I love and hate midjourney
Hi polys,
I wanted to share my concern with you. You all have heard about midjourney, which is extremly hyped. The discord server is insanely crowed and the results of people using the AI are becoming crazier every day. Very realistic results, stylized and anything in between. The AI is so sophisticated that it creates amazing and beautiful artwork following all the common principles that pleases the eye (it uses alot of complementary color palettes for example)
Convolutional Neural Networks
In case you dont know, midjourney is a convolutional neural network, a neural network that works especially well for image generation. And these networks are learning and evolving through many iterations of creating images.
To train a neural network to become this sophisticated, the dataset must be MASSIVE. It probably includes MILLIONS of images.
The datasets are the most crucial for this. Id say 90% of the "amazingness" of midjourney comes from their choice of the dataset. So... they dont just have any dataset of random images, but they must have datasets of artwork.
Concerns
I have my concerns about midjourney and their use of artwork. Ive been asking the devs which datasets they used for developing their network. But they have been intransparent (its a business secret I suppose). They only mentioned a famous image dataset, one of the largest free available ones and stated that other "private" datasets were used aswell.
Where did they get those artworks from?
They for sure did not asked or paid 1 million artists to use their artwork.
What they do is essentially mimicing the styles and creations of other artists.
They do not copy art
It is true however, that the art Midjourney generates is unique. It generates the images from the KNOWLEDGE it gained from the artworks in the dataset. It is still a grayzone to me and it raises questions about where intellectual property of art ends and where interpretation begins.
Artist "Signatures"
It becomes apparent, that they used alot of artists works to train their AI. It even happens that signatures appear, but its never readable. (Midjourney is not really capable of using text in the results)
Assumption 1 - Data Mining
My assumption is, that the devs used data mining techniques (bots will crawl internet pages and download content in reusable and saveable formats. Mostly used to scrape data and save it into a database to create datasets).
This means, they have used sources like deviantart, artstation or forums like polycount, to scrape the artists works and create a dataset full of amazing art.
There is no way, that they could have achieved this level of sophistication and artistic skill with an AI otherwise.
Assumption 2 - Investor bought them a dataset
Another possibility would be that they had an investor who helped them buy a huge dataset of a company like Epic (artstation) or deviant art (owned by wix). This would not make much of a difference in terms of artists right on their own work, but it would be a legal safe call for them, since these platforms probably are capable of using massive anonymized datasets of the artworks uploaded to their platform.
Conclusion
My mind is racing these days. I love midjourney, its incredibly fun. It is a nice break from programming or creating materials substance designer which are both activities with much longer times until you see results. Creating art in midjourney is fun, fast and it just works incredibly well.
But my concerns about artists right rise and since there is no transparency about it, my gut says that something is going on.
Let me know your thoughts in the comments.
Here are some of the results from a few weeks of using midjourney:
Replies
I use Stable Diffusion, and love it. Dreamstudio, the web version of it, is as commercial as Midjourney. But the code is open source, and you can use it at your own PC for free. There is even a Photoshop plugin available already, and other open source software is planning to integrate it too :)
Regarding the dataset, who really cares? It's not that they really stole this material. They crawled for free content. And added some stuff that they bought too i guess.
Let's say though, for instance, you start generating "Photo of a sculpture of a waffle being tortured in hell, made by Jeff Koons"—
Or actually, better example, what if someone sees the work of a small artist, and generates AI versions of paintings "by" that artist, and starts selling them. I think the moral implications ought to be clear that it's using someone else's hard work and stealing potential customers from them. And while it's technically not stealing their actual work, it's stealing their style, that they spent a lot of time developing, in a way that takes very little effort on the end of the AI thief. And then taking that AI art and selling it is watering down / diluting the style brand developed by the original artist.
Of course you could argue anyone can come along and try to imitate a style using traditional methods, and accomplish the same thing— but that still takes hard work, and dedication, and a personal style may well be developed along the path of thiefhood, in a way that's unlikely if you're just typing phrases into AI.
Now, from my perspective, will the AI ever be as good as the original artist? probably not really. But do I think most people will be savvy enough to tell which is real and which is fake? No, I think most people don't care, they just see an image and like it.
But the AI is literally trying to define what characterizes certain phrases / words; so, when you enter in an artist's name, it's trying to puzzle out "what makes this artist's work what it is?" and trying to emulate that. I don't see that as ok, without either robust copyright laws, explicit permission from the artist in question, or laws against making money using AI generated images that derive from someone else's intellectual property (which you could argue would greatly hobble AI art; but, so be it)
it's stealing their style
This part is not copyrighted. And most probably never will. And so it is not stealing. Like you cannot patent a game idea neither.
Pandora's box has been opened and there's no going back, I find it hard to care much about the ethics or legality of using copyrighted works to feed models when the train isn't going to stop. Even if they did it the ethical way with only art that's public domain, it'd still be just as big as a disruption because of how easily these projects are copied. A game studio can take Stable Diffusion, feed in their IP and characters, or even a new character, and have it start generating images with them.
How is published art on artstation "for free content" ? It is intellectual property of artists and should be treated as such.
When uploading to platforms like this, it is obvious that you as an artist agree to the possibility of the artwork being shared on the internet and you (and the platform) have no control over where your artwork lands. But still, does that give anyone the right to use your artwork, your style and craftiness to create their own product?
Imagine this scenario: Midjourney was created by only using ONE artwork. The AI is then capable of generating infinite versions of this, but its still apparently based on this specific artwork.
In this scenario, everyone would expect the creators of Midjourney to credit the artist, right?
So with a dataset scraped from platforms like arstation or deviantart, the only difference is that its thousands of artists whose art is used to create a commercial product. Which makes the intellectual property questino more vague, but it has still the same answer to the core.
I disagree. Why would it not be important to think about ethics? Especially with the level of disruption you describe, it becomes even more important to discuss ethics or legality. I never said that Id like to file a lawsuit. I am a subscriber of midjourney myself and I enjoy their tool, but imo a discord about the ethics is as important as discussing the future of art.
I should clarify, the ethics relating to artists works being used to feed the model I can't bring myself to care about. Purposefully ripping off or imitating art and profiting off of it is still in a similar place as it was before Dall-e, it's just a little bit easier. There are plenty of other ethical issues around AI generated images, particularly biases.
I'm also more concerned what this means for fake portfolios and art tests. And in the near future, what concept artist's role and workflows will be like in the next 5-10 years. And how quickly this is going to bleed into the 3d space.
What do you mean with biases?
Fake portfolios could be a concern, but its also easy to filter out someone who fakes their portfolio through tests.
And yes, I think concept artists are the ones that will have to adjust the most. Iteration speed will increase alot, which is not necessarily a bad thing. :)
It is not your artwork that is published. It is a AI had a look at your image. Then paints in your style. Watercolors for example. Same could have been me, looking at your picture, then use your style. Your content, and everything that you can claim copyright for, is never touched. What you ask for here is that nobody is allowed to use watercolor anymore since this and that artist has used watercolors for his image.
I don't worry for the artists too much though. 99% of the output is plain garbage as i have already found out. And the rest is also most of the times not what you are looking for. It is yet another tool in the pipeline that can speed things up. But not necessary.
You are missing the point and misunderstood my last comment entirely. I didnt say that art generated by the AI and published should not belong to the one creating it using the AI. I am talking about the dataset and the use of intellectual property without consent.
Again, you are saying content on Artstation is "free content", thats what I was referring to.
And when you say 99% of the output is garbage then you are blind: I just screenshotted the community feed for you. Does this look like garbage to you?
Again, you are saying content on Artstation is "free content", thats what I was referring to.
No, i did not say this, and here you misunderstand me. Free would mean that you could grab the images at Artstation and use it for your own needs, put it into videos, make your own images with it ant so on. Like CC0 content. But the AI does not break any copyright by just looking at your images and analyzing it. As told, the same does every other artist that visits Artstation. Or by visiting the Louvre and studying the old masters. Or by simply looking out of the window. (Crazy that you are not allowed to make a picture of some buildings anymore)
And when you say 99% of the output is garbage then you are blind: I just screenshotted the community feed for you. Does this look like garbage to you?
I talk about personal experience. I try to use SD images at the moment to make a video. What you show is the 1% of useful content after lots of refinement steps. But from this 1% it's again just a fraction close to what you wanted to achieve. It's throwing a dice. There is not this much control over the result. The same terms will give you always a different image.
Yesterday i have created around 1000 images, and just a handful looks not completely destroyed. And i have yet to find the image that looks exactly how i wanted it to be.
This is the average result when you try to create a happy woman in a red dress. The 99% garbage. Midjourney is a bit better. But not this much. As told, i don't worry about artists at this point ;)
Saying that 99% of the output is garbage makes no sense.
It is a work in progress and its part of the AI generation process to reroll and iterate & prompt craft to get the results you want. It takes time and iterations. Just like any other process creating art does.
Its the same with sculpting.
Does the image below look like an epic dragon skull? no. because its not done yet. Same for the images you sent.
So yes, the screenshot of the community feed is representitive of the quality that midjourney can produce. You dont measure ZBrush's performance as an art tool through the visual quality of work in progress sculpts, do you?
Well, I already provided proof for my experience :)
Most of the times there is no further refinement possible. The prompt allows just so and so much keywords. And the AI has its own mindset. I agree that with the right keywords you can get good looking and even awesome results. But most probably completely different from what you want to achieve. It's like, hey, i wanted to create a space station, but this cool cybercity is much better.
Try to create a girl face with wrinkles under the brown eyes, the flowers in her hair in another color, the eye brows a bit higher than in the former iteration, and so on. The things that you call refinement. It will not refine, the image will change completely. It's not that it grabs the existing face and adds the missing bits. Not even with image to image ( this requires a good looking base image already, so you are again trapped) . And looking to the left or looking to the right is something that the AI completely ignores most of the times. I had a hard time to let a human at least face the camera. I always ended in a back view.
Working with the AI solutions is not like modeling and refining until you are happy. You throw a dice. And hope that a useful result is included. And then you throw another dice. And another one. And as told, here 99 of 100 images is simply garbage. Maybe we can say, the results are still fine, but useless for what you wanted to achieve.
Your dragon, finished or not, is at least to recognize as a dragon at any step. While with SD, Midjourney and co you get 99 images of 100 that doesn't even look close to a dragon. I know what i talk about, i also had my fun with dragon images already.
Did you consider that you just dont know (yet) how to use the AI properly?
There is a random component, yes. But you can very well prompt craft and refine until you get the results you want. I was able to combine various elements that I wanted to see. The more specific and the more elements you want, the more difficult it will be, but its not impossible to get exactly what you are looking for. It obviously has less control over the result than a traditional digital artist.
But your assumption is just wrong: You provide proof for your own methods result in 99% bad images, but how does that transfer to everyone that uses AI to generate images and to people who have practiced using the AI?
The quality and "likeliness of closeness to your idea" of your artworks mostly based on your skill of prompt crafting:
+ using the right words
+ using the right combinations (style combinations, word combinations)
+ considering prompt order
+ text weights
+ image weights (style transfer)
+ right ratio for the idea (portraits work better in --ar 2:3 for example)
I recommend you to check out this extensive collection of keywords that will help you to not get 99% bad results, trust me. :)
https://github.com/willwulfken/MidJourney-Styles-and-Keywords-Reference/blob/main/Pages/MJ_V3/Style_Pages/Colors_and_Palettes.md
Did you consider that you just dont know (yet) how to use the AI properly?
:D
There is a random component, yes. ... It obviously has less control over the result than a traditional digital artist.
That's what i talk about. The general concept behind AI art. Not if i am just too dumb for the job :)
It's not you paint the rough structures and then refine. It's throwing a dice. With every throw you might get a bit closer. But it can also be the exact opposite. That you move away from what you want to achieve.
I recommend you to check out this extensive collection of keywords that will help you to not get 99% bad results, trust me. :)
Not to forget this one, which i use in conjunction with Stable Diffusion: https://lexica.art/?q=drum
Which should answer your question if i know how to use it. I get my results.
But again, i define a bad result when it is not what i am after. And of course the really bad results when the AI simply doesn't want to do what i want it to do. Everything around humans is still problematic. Just search through the results at lexica.
And some things are simply not to achieve. Ladybugs for example ... https://lexica.art/?q=ladybug
Deviantart, artstation and others should allow filtering AI-made works as option, to prevent mass confusion.
Perhaps even make a separate subsite for that.
Some platforms like Getty forbids AI art already ...
https://twitter.com/mediagazer/status/1572596647774887936
FWIW, here's where things are at the moment a little more than a month later : writers are developping full-on comic books using MJ for illustrations.
https://campfirenyc.com/comics/
"The Lesson" and "Exodus" are probably the most visually striking.
Of course the images probably took a lot of attempts to land just right in accordance to what the writer wanted (which in and of itself is a process I find quite unfulfilling and absurd to an extent, compared to the pleasure of solving a visual problem on ones own through the act of sketching) ; but IMHO it would be foolish to pretend that a singularity point hasn't been crossed in terms of objective metrics of quality, especially when factoring in speed.
Now I am not saying that these comics being a reality it is a good thing or a bad thing, or that the process to train the AI was art theft of not, or that this new context is "fair" or "unfair" to regular illustrators looking for gigs (as no one is ever owed a job anyways : everyone has to adapt to technological leaps faster than ever before). And after all there is nothing new about being able to produce an illustrated version of a story without having to hire illustrator, as this has always been possible using photography and clever collages (or even crude stick figures). But I just think this example really shows that it is foolish to think that "it will never look good enough". The tool can already perfectly emulate classic illustration styles and bypass the need for an artist altogether at least for the production of a graphic novel.
One interesting question will be whether or not the public will be interested in such products while knowing that a massive component has not been human-crafted. To be perfectly honest it only took a few pages for me to completely forget about it ...
I guess the next step will be to see if people are willing to actually pay for that kind of stuff, or if it will be quickly considered as cheapo, disposable free entertainment. I can imagine that some people attached to the craft will straight up boycott commercial AI-generated content, while 99% of the people out there will be happy to consume it and even pay for it. But since we'll see hundreds of AI-drawn comics similar to the above in the coming months ... very few of the (human) writers releasing them will likely ever make any profit out of them or strike publishing deals as there will be gazillions of them in no time.
Perhaps this will lead to a new rise of highly stylized 2d art, since the more stylized and simplified the art the faster the pages are to make. Which isn't too different from "bad art" mangas like One Punch Man and Attack On Titan, which weren't hindered in any way from their art being a bit on the crude side.
One thing for sure, this makes me appreciate the craft of the best classic comics/graphic novels in my (very physical) book collection more than ever before as it somehow adds an extra layer of nostalgia and fragility to them when I find myself wondering if such wonderful pieces of illustrative art would have even existed had art AIs been already a thing back then.
Nah...that ai thingy was bang on, looks like a masonic handshake :p
I have some thoughts regarding this that I just kinda feel like writing out, and polycount seems as good a place as any to do so. This is gonna be kinda long and rambling and I'll apologize for that up front, just feel like writing it. Picking this thread as I find @pior 's comic link as a particularly up-to-date example, and because I'm mainly looking at this from the standpoint of a 2D artist.
Firstly, just to get it out there, there's definitely an element to this that I find frightening. To an extent I find it insulting - not that people can generate art. Variations of "generate art without much thought" have existed for a long, long time. But the nature of this - effectively relying on and being built off the backs of artists who did put in the time, who did define the styles - to know now that people with minimal training and study can and will claim it's their imagination, their design... makes my stomach churn. I'm not all that interested in gate keeping, and I think there are ways to utilize AI generation while effectively retaining creative authorship - but let's face it... That won't be the case, it won't be something people are honest about, and it will diminish the value of art creation as a whole.
I do also think there's some pretty enormous gravity to these recent AI developments, and it seems - frankly, that they're moving faster than the world is able to keep up with. I know of several companies already looking to see how viable using AI generated art is to handle certain tasks - and I'm quite sure in the very near future, legal departments either are already or will be going apeshit over the ramifications of not knowing where the art actually came from. Is that dress a designer dress that now we're claiming ownership over the design for "because AI did it"? Is that weapon something that might be mistaken for an iconic concept in a popular video game, that you just aren't familiar with? Did it slap some iconic landmarks you're not allowed to depict in that cityscape in the background? The inability to claim you've done your due diligence in developing the design yourself, not knowing where it pulled those design elements from... that's gonna be a fucking shit show. And people are trying to run with it already anyway. I expect we'll have some noteworthy legal hurdles to try and navigate these issues, the issues of sourcing the "learning" material, etc.
On the topic of this "replacing" artists - I think it's ignorant to say that this doesn't hurt artists, that it's just a tool, and that the results aren't able to replace the work of actual people. From what I've seen develop just over the past couple months, it will absolutely hurt some artists, help others, and in the process - completely change the way the modern world thinks about art and asset creation. I think the main folks this is going to hurt initially are:
I think this will affect almost everyone working as an artist, 2D/3D/animation/whatever. Am I personally worried about losing my job? Hahaha, no. Technology moves quickly - the ability of people, teams, companies, and workflows to adapt as quickly on a large scale is not so swift. Also, consistency and staying on-brand/design is HUGE, and AI currently sucks at that (those comics are a great example, with space suit, VR helmet, and monster designs being utterly inconsistent throughout). Knowing how to implement the shit you're making, how it needs to be used, and being able to make adjustments to it is like 80% of my job and that's not going anywhere. Do I think within the next few years, I'll need to adapt to this technology - and investigate (pending legal hurdles) how I can utilize it to optimize certain things that my team does? Yes. Yes, definitely.
I do think this might mean there are less artists in the coming years (and yes, I am presently defining "artists" as people who like, actually make the shit). There's been a pretty significant influx of artists over the past ~15 years or so I think due to the success of video games, and entertainment media in general - and this is going to "weed out" a lot of those who don't move beyond a certain point of competency. Used to be anyone who was even mediocre at art could say "Well if you don't want to pay me, do it yourself" - and now, most people could. A lot of positions will move from "creation" to "clean up", and the allure of pursuing art will - I suspect, dissipate for many people.
I guess finally, I will say that while there is a scary element to this, even a very very sad element to it, I guess I'm not too worried. I didn't learn art for the money, I did it because it felt like what I was supposed to do and there is shit I want to make, that I want to design. I am fortunate to work as an artist, but asset creation is a small portion of my value. And if being an adult has taught me anything, it's that a lot of people are very, very bad at their jobs so there's always work to be done. At the end of the day, I just want to spend time with my wife and dog, pay off my house, and make the things I want to make. Whether I make my money illustrating things myself, cleaning up AI-generated BS, or just telling other people "looks good", "looks bad" - what does it matter? I don't owe the world my art anymore than the world owes me a job.
Still just one foot in front of the other, reacting to whatever's up ahead.
I very much love the work of Mohammid Qasim Iqbal
https://parametric-architecture.com/the-rabbit-hole-of-generating-purely-fun-and-harmless-images/
Well, I am very curious to know more about the licensing terms for such "harmless" images (which are dead easy to produce with just a simple access to the MJ discord), and the potential repercussions of it all as outlined above by @Two Listen. It's such a mind boggling spiral : these are super easy to make, therefore no one will ever be interested in hiring this person prompting them ; and if they end up being used for a commercial design, then sure enough MJ owns part of it.
And without a doubt, images from famous firms like Zaha Hadid were likely used in the training model. Really can't wait for the unavoidable lawsuits that are bound to come soon.
It's also interesting how much such images "cheapens themselves" instantly so to speak. Had this guy been making similar images a few years ago on his own using photomanipulations and Grasshoper renders, he could have been able to make a name for himself doing so. But these being AI-generated basically reduces his impact to 5 minutes of online fame and just another Instagram account pumping AI images. Beautiful images ... but with close to zero market value. Fascinating stuff.
Well, the market is still there. It has not gone away just because there is now a flood of images. There always was. It has just become yet again a bit tougher to find the niche.
I think it really comes down to the end use case.
So far, seeing these images provides a good reference point for inspiration, or even concept art for 3D art creation.
Don't know how they would be directly used to create something consistent, not to mention that MJ or stable diffusion can't generate a character art sheet coherently.
Like if I wanted to make a comic book, its impossible to generate panels precisely since each output is quite different. So unless its a random comic full of nonsense, there's a lot of work to correct inconsistencies.
It can be used to generate hype though, like the zelda movie one, so essentially seems more a marketing tool.
I'm glad that many are using it to create something that they see some value in even if it ends up as a few seconds of attention seeking on social media.
Not sure how companies would legitimize it, I expect when the algorithms make their way into major software like Photoshop then they can use it more legally and the data sets would also be more legally insulated.
But I still see it as the most early reference given to a concept artist by someone not skilled in art but maybe in ideation in production at a larger studio.
Book cover artists are probably screwed though, this technology really does impact work that requires non repetitive single use creations, like one illustration for a card and that's it. You don't need an artist to start that if you're willing to go through a lot of garbage generated by AI to get the result you're willing to settle for.
AI artists are pretty much DJ's.
Don't need to be a real musician to be a DJ, but still need to understand the fundamentals of how music works.
Hard not to see this being massively disruptive for what we do, until we stop being utterly derivative in the kinds of art we make.
Its the speed at which its evolving that unsettles me. Like I'd seen this compilation of baby and toon versions of marvel characters and the AI was able to capture subtleties of personality in the variations.
Credit: https://www.facebook.com/media/set/?vanity=metzae&set=a.557890323008966
I think knowing fundamentals comes down to the knowledge you need to write the appropriate prompt especially being aware of an artists name/handle/style that AI can derive from.
Does make it easier if you don't concept or want to use AI as a refining tool.
The generic workflows in AAA atleast for 3D artists seems stable enough since the output is derivative from the concept art but I worry for concept artists, though they might turn into clean up artists where the lead uses AI to generate an image and the rest of the team cleans up the output, so maybe a bit like plate cleanup in film.
It is encouraging that it would motivate more people to become independent creators since pitching concepts becomes a whole lot easier.
The generic workflows in AAA atleast for 3D artists seems stable enough since the output is derivative from the concept art
Are you sure? :D
https://www.kaedim3d.com/
Voila, here comes the midjourney to 3d model pipeline :)
AI will for sure change the industry. But up to now the changes has always raised new opportunities then. I am and remain optimistic that it remains this way :)
There's a lot in the comments that suggests that that program with an outrageous subscription price is a scam and the models are being made by actual artists.
Though wth regards to stability for 3D artists, I am thinking it comes down to what you do in the studio and how creative your work is.
In AAA for instance from my experience and colleagues in other studios the amount of creative work they do is very little, meaning not that they don't do any, if there is a faster way that is what is usually prioritised and it can become tedious and repetitive.
For example use of asset libraries and modular sets for environments, outsourcing unique assets, focusing on population of a level rather than actual asset creation. Really depends on the project.
Thinking most of AI will go towards automating these tasks first which is a good thing.
I still prefer using AI for generating a reference mood board and don't have much commercial use for the unaltered output.
Custom Trained AI is doing some interesting stuff already.
https://twitter.com/emmanuel_2m/status/1589995198289182720
Well, even when it really should be scam, you don't need AI to create a mesh out of a photo. Photogrammetry software can do this job since years.
Amazing times. I really look forward how the world has changed in one or two years :)
Once computers learn to draw hands, 50% of artists will be out a job
Nah, there are still feet ...
The thing I find fascinating about the quotes from the person pumping the potion example is how they show a complete misunderstanding of game design - or rather an attempt at wrapping/bending the narrative to fit the concept of an infinite art generator. Because outside of the most mindless games targetting a demographic of always-on addicts, no game (or narrative piece in general) benefits from an infinite amout of visual content. A Link To The Past never needed more than a single jar graphic with a few palette swaps and one single variant ; and TF2 didn't need an infinite number a character designs in order to give flesh to a team of well balanced classes. And so on ...
And if an art director needs an AI to get "inspired" or have a vision ... then that's probably not an AD one would like to work with/for :D Over-excited loudmouth "idea people" are truly the worse kind of people to work with, and these tools will make them all the more insufferable. Like telling concept artists to rework this or that design based on a set of AI designs they "prompted" the night before.
Now that we are way past the point at which this content is undistinguishable from highly polished concept art, IMHO the best games/movies will not come from people who can "craft the best promps" (as this notion alone is completely ridiculous since litterally anyone can do that, it's not rocket science really) but rather from those who have a traditional art training and therefore are aware of concepts like line economy, the use of suggestive shapes, minimalist designs (as it has always been the case really) ... and likely no desire to use AI-generators at all. Because as amazing as the tech is, I can't imagine the atmosphere at a game studio being healthy if all the designs are being "AI-prompted" by a bunch of idea/marketing people or a single pseudo-AD as opposed to being produced by a team of real humans that everyone on the team can interact with and bounce ideas to ...
All that said, the incoming tidal wave of "Marvel Babies" level garbage will sure require some bracing before impact because that stuff will be everywhere in no time.
Oh by the way : there have been many talks about how artists would have much preferred to have been asked for consent beforehand to allow their work to be fed into a rogue AI training model. Of course it's too late for that now as models have been heavily trained on ArtStation content and famous artists already.
From there I'll submit the following : every single 3D model currently uploaded on Sketchfab is probably already being used to train 2D to 3D AIs, since generating hundreds of screenshots from the viewer is trivial. So any artist thinking that they made a niche for themselves doing beautifully crafted handpainted models based on concept art is likely going to have to deal with the fact that their specialty will be fully replaced by a clever AI generator within 6 months or so. And since this craft is derivative by nature, even fewer people will feel sorry for it to go away.
Kinda sucks people will likely no longer upload easily scraped 3d models to the internet. I wonder if someone is going to try to scrap 3d assets from games and use that to feed a model.
Well ... every game out there is already very easy to extract content from, especially if it is done in Unreal Engine. This has always been fantastic for studying purposes but this also means that this content is perfectly ready for AI training to chew on today. I'd say that perfectly usable "modelsheet turnaround to mediumpoly models" generators are about 6 months away or so from now.
Son of a biscuit. More thoughts!
You know, I've never had a drink in my life. Perhaps this is how I start drinking? Only time will tell, but if anything was going to be that final straw...
"idea people"
Don't you hate them :D
But why do you assume that everybody who uses AI to create content must be completely clueless about graphics? I know more than one artist who has simply adopted the new tool into his workflow already, me included. The artistic skills are still needed. Somebody has to judge if the result is good enough. Once the hype is over you will see lots of quality work again. Currently they sink a bit in the masses of the enthusiasts.
Books did not make hand written letters and manuscripts go away. Photography did not make the traditional drawing go away. Movie did not make the photos go away. And AI will not make all 2d or 3d art go away. These techniques all made the world richer, not poorer.
See the chance :)
Heya - I don't think anyone in this thread implied this. As far as I am concerned my point is that it will without a doubt make some creative work environments very toxic, very fast. To sum up my above points I can see this happening in two ways :
1 - Loudmouth "idea people" (either high up in the executive ranks or from marketing departments) feeling like they have just as much of a say than an AD or a creative team on the way a project looks, just because they can "prompt" images instantly. Of course one could argue that this is just like any disruption caused by tech : teams and production environments will have to adapt, that this is just "democratization of art", and so on. But that's not my point - I am talking about what can/is happening *right now*, not years down the line.
2 - Also, taking AI generation as it is right now without making any further predictions (i.e. it being able to generate polished one-shot concept art, a thing that was considered impossible by many just a few weeks ago when this thread was originally created), a direct logical consequence is that 3d modelers can already be handed down designs that have only taken a "prompt" to generate, for them to slave at for 2 weeks/a month. That means no human face to associate with the design they are told to recreate in 3d, no one to talk to about the inspiration for this or that aspect of it, no interesting chat about the studies and mastering of craft that allowed the designer to produce this well-balanced design, and so on. This may not seem like much to "idea people" only operating based on what was recently popular in the latest Marvel movie, but these interactions within an art team are IMHO the glue that makes an art team work well as a cohesive unit and builds up the feeling of a shared culture emerging from a project. Removing this can make everything fall apart, and I feel like any AD with solid experience under their belt will recognize that.
- - - - -
As for equating AI generators to past optimizations and process improvements allowed by tech, like the invention of the printing press : I don't see much of a point discussing this as this is purely subjective and (I suspect) mostly brought up as some sort of "gotcha". My points above are not about this or that aspect of the tech being good or bad - only framing the tech through the lens of the psychology of the people using it aggressively, because of the instant kick it gives them to prompt "a cool sexy character trending on Artstation in the style of Hideo Kojima, super ultra good and realistic in 4K", and ultimately pushing this onto the fragile creative environment of an art team.
Also : there are without a doubt some lawsuits already brewing, since the likeness of actors and unmistakable character designs were obviously part of the training model that was used to generate the stylized Marvel characters above. I don't think it will be very hard for Disney lawyers to argue that the unlicensed use of their data as part of a ML traiing set is illegal, especially with the precedent that was set by audio ML models requiring an opt-in specifically to avoid being sued to oblivion by music majors, and the exemption that is usually granted to research projects in contrast to commercial ones ...
I just don't see how things could not get very messy from there.
I hear your concerns. I even agree to some. I just don't have this much to say to the points where we agree :)
1) They always existed. That's teamwork. One is doing the work, ten stands around and gives the good tips :)
2) That's 3D. 99% is not creative and never was, but simply repeating boring steps and following what is already given. This was also the case before AI. And i don't see how it could impact the output if the concept is made of a human or AI.
AI will have an impact for sure. In the end time will tell. Where we disagree is what impact it will be. Fact is the AI solutions are out since many months. Not just since Midjourney, DallE and Stable Diffusion went live. This started much earlier with tools like Crayon(Dalle Mini). And the world did not stop to rotate yet.
Regarding the legal part, you can also make copyrighted material with Photoshop. It's not the tool. The artist is still in charge here.
This will obviously have an impact on artists - so of course some of the people involved will be artists, just doing their best to do what they've always done.
And while I appreciate and respect Pior glancing over this portion - I'm gonna disagree with your last bit. Like Pior said, it's subjective, but this is not like past technological advancements and at least personally I make a key distinction: The camera was not built from the works of artists from all corners of the world without their consent. The printing press did not require unconsenting writers across the globe to submit their works to succeed. Hatsune Miku did not pull the voices of countless unaware singers - novice and professional alike, to function.
And yes, that's where I personally draw a line. I'm sure we'll have folks who can do the work, who will create the training material they feed in and use it to further the scope of their own work. I know tons of people will use it for composition ideas, an alternative to google images, etc. The idea of AI learning does not disgust me at its core, and I believe it can be used as a tool. But this makes the water...muddy. Used to be even things I didn't much care for as a product would at least give me pause, or could give me some measure of joy because of the art. I liked seeing work from other artists out there, to look over the cards for some tabletop game, a random bookcover, or some handpainted box art for something I've never heard of and to know that it was crafted. I liked seeing their names, looking up their artstations, or guessing if I knew who did the work prior to checking! And I liked knowing that I could admire the work involved - the time, study, subtlety of decisions made.
I'm not expecting that to up and disappear. Those things, I'm sure, will still be there. But where previously they'd catch my eye from a distance, a few years from now I might have to fish them out of some real... muddy waters.
As you said, time will tell.
I can't believe this is happening...this job is what gave me confidence in myself. It would crush me if AI replaced me...I also really want to learn 2D art but why bother? It's gonna take years to learn and who knows how good this damn AI will get?
We're gonna reach a point where all of us will be just mindless consumers consuming regurgitated "good enough" art. There will be no mountain to climb, no skills to master. It's like I made a program that beats the game instantly and then repeats it ad nauseam. I treat art and game development like I am a video game character doing quests. What fun is there in instant results? Even if you invent a new art style, it will be just fuel for the AI and pretty soon your results will be buried but derivatives.
I can see the end game, in 10 or so years. It will be thought-driven metaverse interfaced through a brain implant that reads our thoughts and makes us dream whatever we want. No typing required. Want to be in World of Warcraft? Just imagine it and the AI will do the work for you. Want to filter out the stream of consciousness thoughts that are irrelevant to the fantasy? Just tell the AI which thoughts are relevant to the simulation. It's going to harvest your thoughts and tailor experiences for you. Everything's going to be artificial, fake, vapid vomit.
This AI is just a parasite that feeds on humanity...I hate everything it represents and I hate the people who are working on this. Truly.
I can't believe this. I really can't. I can't stop thinking about this.
@Tiles : while I do believe that your points are being made in good faith (and to an extent from the position of the devil's advocate, which is always healthy) the position of "it's always been like that before AI" just doesn't hold at all. Quoting you :
"2) That's 3D. 99% is not creative and never was, but simply repeating boring steps and following what is already given. This was also the case before AI. And i don't see how it could impact the output if the concept is made of a human or AI."
There's more to the "output" than just the image or the model. The point I am trying to make is that injecting fully AI-generated designs into the creative process of an otherwise healthy art team is IMHO a receipe for absolute disaster in terms of team cohesion, and that's something that no team leader worth their salt will be willing to take (while over-excited "idea people" will jump at the opportunity of course). Because even though the job of a 3D modeler is indeed no less derivative than that of train model maker, there is a huge compensation coming from being able to interact with the person who crafted a design, or, inversely, from being able to interact with the person being tasked to bring ones design to life. As a matter of fact and as someone who's been on both sides of this dichotomy I would say that this sharing of knowledge and human interaction is by far the most rewarding aspect of the job, in both cases. Without that there's really not much left.
Of course and as stated earlier, no one is owed a job and everyone is free to leave if they don't like the process in place at a studio, as this sharing of knowledge does not *have* to be there. Again my point is not that it is either good or bad, or legal or not - I am simply following/extrapolating the curve and submitting scenarios that I see coming from a mile away, for the sake of the discussion. The TLDR of my point is that regardless of it being considered legal or not, injecting that stuff aggressively into an art pipeline is a huge (and somewhat pointless) risk to take.
- - - - -
As for the position of "it's just a tool like all other tech advances before", as said I didn't see much reason for discussing it originally because it easily leads to absurd arguments along the lines of "if AI generators should be banned, then we should ban web browsers too, because they can display pictures or Disney characters just the same, lalalalala" and I am just not interested in going there. In the case of the Vizcom sketch-to-render engine, perhaps the "it's just a tool" argument can hold ; but in the case of MJ and Stable diffusion image generation I don't think it will be very hard for Disney lawyers to blow through that line of defense if a lawsuit was to take place.
Furthermore and setting aside the obvious unethical nature of a training model built on images without any prior opt-in consent, the people making a commercial use out of these training models (like how MJ has paid tiers) IMHO already painted themselves in a corner. MJ curating their so-called magical "black box" to avoid child pornography and gore images from being generated (because of the fear or the obvious legal repercussions) and Stable deciding to go for an opt-in system for their music ML tool because of the fear of music majors suing them to oblivion proves that the tech is not a neutral tool in any way but rather a curated entity, which has been trained on the work on unknowing artists who never gave their consent even though it didn't have to be. This is IMHO somewhat similar to how Social Media platforms will soon enough have to take a stance between being a publisher (curating content as they please) or a neutral communication tool. It may not be illegal now, but I can't see this not being regulated soon.
What a mess !
people used to fling stone spears at mammoths to get meat.
how lame and pathetic our lives would seem to them if they could know how we live now.
No matter what tech is involved with making games, always going to be some difficulties in making them and selling them. Whoever can solve the difficulties will remain valuable.
Technology will continue to devalue humans and make it harder for anybody not in the ownership class to live. Artist aren't under unique threat in that regard. I don't think it's wise to wrap up our happiness and ego in our work. That's something that will always need to be changing.
Yes, i play the devils advocate here, to some degree. Sorry ^^
... obvious unethical nature of a training model ...
It is in the same way ethical and legal than a robot replacing a worker in a factory. In the end the ethic question doesn't even matter. It is about if it is legal. And about if you are still be able to compete with paying 20 people while the robot can do the same job, just more accurate and for fewer money.
... but in the case of MJ and Stable diffusion image generation I don't think it will be very hard for Disney lawyers to blow through that line of defense if a lawsuit was to take place ...
Again, you can paint any of these characters also in Photoshop and model them in Maya. Shall Photoshop and Maya now come with a content filter? Or be forbidden since you can do illegal content with it? And shall we forbid a scissor and copy and paste since with that you can cut out copyrighted material and use it in your own images? And what about brushes? With brushes you can paint illegal content !
Yes i know, it's the kind of question that you don't want to answer. It is indeed a special rabbit hole. But it is the exact question that needs to be answered. This is the casus knaxus. It's the job of the artist / publisher to just release legal image material. Not the job of the tool to censor away anything possibly illegal. For my personal taste SD, DallE and Midjourney has already overdone the self censoring.
There is no way to sue SD, DallE or Midjourney. They didn't do anything illegal. To crawl the net for freely reachable images is allowed where they live. To look at images and analyze them is as told something that every artist does since humans exists. And it is not forbidden to generate images. And that was it already. Nothing illegal happened. There is nothing left to sue them. Guess why after months there is still not a single indictment.
What you can sue for is for stealing copyrighted material. Design, brands, there must be license breaches. But this needs to happen at a one by one case. And guilty is the one who has put it into the public, not the tool. To type in an artist name or a character name into a prompt will never be part of a lawsuit i fear.
As told, i see your fears and concerns. And i partially share it. It feels highly unfair when you spend four weeks at an image or a game asset, and somebody else just presses a button and has a even better result. I also know that my opinion is biased. I have fallen in love with these tools. But either way, you cannot stop the future. The AI spirit is out of the bottle now. The only question remaining is how to deal with it. I personally cannot see AI as an unethical or evil tool. But simply as another tool in the belt. In an always rapidly changing world anyways.
I once started with drawing by hand. Then the PC arrived. At the PC i started with pixel art. Then came 3d with box modeling, pushing vertice by vertice around. Woah, SDS! Woah, you can select edgeloops instead to select the verts one by one? Crazy shit! Woah what a technical jump it was to start sculpting. And geez, Normalmaps, PBR ... \o/
The only constant in life is constant change. All you can do is to adopt :)
Hi there !
To be clear, on "Yes i know, it's the kind of question that you don't want to answer" (in regards to the It'S jUsT a ToOl point) : my position is not that of not wanting to answer because the topic bothers me or because I don't see the argument as winnable ; I am simply not too interested by it this specific aspect of the discussion because even though it is front and center to all discussions about AI art generation, I am just not interested in putting too much energy there - it's simply a resource allocation thing :D
Now one thing to keep in mind still, is that even though one can't be condemned for an act that wasn't defined as unlawful at the time of the act (in well-functioning countries at least), law is still subject to the interpretation of the judge and as said I don't think it would be too hard to build a case against image scraping for ML without consent (especially since a frame of reference has already been set, with the allowing of non-commercial research projects to do such wide scraping ... because they're non-commercial). But of course the allowing of something doesn't mean that the fuzzy opposite is illegal. At the end of the day even is such hypothetical lawsuits don't succeed, they could still highlight some very interesting things and the need to adjust copyright law. But then it could also turn out completely unexpectedly, with even worse consequence than the current state of things ...
Ironically, while I still believe that the tech is based on a deeply unethical practice I also find the current state of things somewhat motivating. Not so much because of the potential of tools like MJ (as I find the overexcited hype around them and the constant stream of output to be both quite nauseating), but rather because once the initial shock to the ego wears off, it then forces one to be proactive about ones approach to art, life and career goals, and so on.
You guys are way to excited about this thing its either just another social engineering experiment or get rich scheme Just have to follow the general trend of everything failing after initial hype -> shit coins failing NFTs failed sealed games failed fake gurus failing metaverse failed ... add your favorite here
5 years ago was VR and that this is almost dead now back then 7-80% of all freelance jobs was VR related now you cant find one even if you look for I guess all this new stuff are failing because they are build on very shaky ground and have very little actual value digital stuff doesn't exist sealed video game doesn't cost millions and so on
The things is that this generators haven't even start making money so when they do some big company will either buy and shelf them or weaponized them
Either way we have live with more stupidity in our lives
There is no "tech" in this.
Using ai generated art is same as downloading shit from clipart stock websites.
It's basically "i'm not an artist so i request it instead".
So, yeah. That hurts our souls :(