Home General Discussion

I love and hate midjourney

greentooth
Offline / Send Message
Finnn greentooth

Hi polys,

I wanted to share my concern with you. You all have heard about midjourney, which is extremly hyped. The discord server is insanely crowed and the results of people using the AI are becoming crazier every day. Very realistic results, stylized and anything in between. The AI is so sophisticated that it creates amazing and beautiful artwork following all the common principles that pleases the eye (it uses alot of complementary color palettes for example)

Convolutional Neural Networks

In case you dont know, midjourney is a convolutional neural network, a neural network that works especially well for image generation. And these networks are learning and evolving through many iterations of creating images.

To train a neural network to become this sophisticated, the dataset must be MASSIVE. It probably includes MILLIONS of images.

The datasets are the most crucial for this. Id say 90% of the "amazingness" of midjourney comes from their choice of the dataset. So... they dont just have any dataset of random images, but they must have datasets of artwork.

Concerns

I have my concerns about midjourney and their use of artwork. Ive been asking the devs which datasets they used for developing their network. But they have been intransparent (its a business secret I suppose). They only mentioned a famous image dataset, one of the largest free available ones and stated that other "private" datasets were used aswell.

Where did they get those artworks from?

They for sure did not asked or paid 1 million artists to use their artwork.

What they do is essentially mimicing the styles and creations of other artists.

They do not copy art

It is true however, that the art Midjourney generates is unique. It generates the images from the KNOWLEDGE it gained from the artworks in the dataset. It is still a grayzone to me and it raises questions about where intellectual property of art ends and where interpretation begins.

Artist "Signatures"

It becomes apparent, that they used alot of artists works to train their AI. It even happens that signatures appear, but its never readable. (Midjourney is not really capable of using text in the results)


Assumption 1 - Data Mining

My assumption is, that the devs used data mining techniques (bots will crawl internet pages and download content in reusable and saveable formats. Mostly used to scrape data and save it into a database to create datasets).

This means, they have used sources like deviantart, artstation or forums like polycount, to scrape the artists works and create a dataset full of amazing art.

There is no way, that they could have achieved this level of sophistication and artistic skill with an AI otherwise.

Assumption 2 - Investor bought them a dataset

Another possibility would be that they had an investor who helped them buy a huge dataset of a company like Epic (artstation) or deviant art (owned by wix). This would not make much of a difference in terms of artists right on their own work, but it would be a legal safe call for them, since these platforms probably are capable of using massive anonymized datasets of the artworks uploaded to their platform.


Conclusion

My mind is racing these days. I love midjourney, its incredibly fun. It is a nice break from programming or creating materials substance designer which are both activities with much longer times until you see results. Creating art in midjourney is fun, fast and it just works incredibly well.

But my concerns about artists right rise and since there is no transparency about it, my gut says that something is going on.


Let me know your thoughts in the comments.


Here are some of the results from a few weeks of using midjourney:

Replies

  • Tiles
    Online / Send Message
    Tiles polycounter lvl 10

    I use Stable Diffusion, and love it. Dreamstudio, the web version of it, is as commercial as Midjourney. But the code is open source, and you can use it at your own PC for free. There is even a Photoshop plugin available already, and other open source software is planning to integrate it too :)

    Regarding the dataset, who really cares? It's not that they really stole this material. They crawled for free content. And added some stuff that they bought too i guess.

  • Joopson
    Offline / Send Message
    Joopson Polycount Sponsor

    Let's say though, for instance, you start generating "Photo of a sculpture of a waffle being tortured in hell, made by Jeff Koons"—

    Or actually, better example, what if someone sees the work of a small artist, and generates AI versions of paintings "by" that artist, and starts selling them. I think the moral implications ought to be clear that it's using someone else's hard work and stealing potential customers from them. And while it's technically not stealing their actual work, it's stealing their style, that they spent a lot of time developing, in a way that takes very little effort on the end of the AI thief. And then taking that AI art and selling it is watering down / diluting the style brand developed by the original artist.

    Of course you could argue anyone can come along and try to imitate a style using traditional methods, and accomplish the same thing— but that still takes hard work, and dedication, and a personal style may well be developed along the path of thiefhood, in a way that's unlikely if you're just typing phrases into AI.


    Now, from my perspective, will the AI ever be as good as the original artist? probably not really. But do I think most people will be savvy enough to tell which is real and which is fake? No, I think most people don't care, they just see an image and like it.


    But the AI is literally trying to define what characterizes certain phrases / words; so, when you enter in an artist's name, it's trying to puzzle out "what makes this artist's work what it is?" and trying to emulate that. I don't see that as ok, without either robust copyright laws, explicit permission from the artist in question, or laws against making money using AI generated images that derive from someone else's intellectual property (which you could argue would greatly hobble AI art; but, so be it)

  • Tiles
    Online / Send Message
    Tiles polycounter lvl 10

    it's stealing their style

    This part is not copyrighted. And most probably never will. And so it is not stealing. Like you cannot patent a game idea neither.

  • ZacD
    Offline / Send Message
    ZacD sublime tool

    Pandora's box has been opened and there's no going back, I find it hard to care much about the ethics or legality of using copyrighted works to feed models when the train isn't going to stop. Even if they did it the ethical way with only art that's public domain, it'd still be just as big as a disruption because of how easily these projects are copied. A game studio can take Stable Diffusion, feed in their IP and characters, or even a new character, and have it start generating images with them.

  • Finnn
    Offline / Send Message
    Finnn greentooth

    How is published art on artstation "for free content" ? It is intellectual property of artists and should be treated as such.

    When uploading to platforms like this, it is obvious that you as an artist agree to the possibility of the artwork being shared on the internet and you (and the platform) have no control over where your artwork lands. But still, does that give anyone the right to use your artwork, your style and craftiness to create their own product?

    Imagine this scenario: Midjourney was created by only using ONE artwork. The AI is then capable of generating infinite versions of this, but its still apparently based on this specific artwork.

    In this scenario, everyone would expect the creators of Midjourney to credit the artist, right?

    So with a dataset scraped from platforms like arstation or deviantart, the only difference is that its thousands of artists whose art is used to create a commercial product. Which makes the intellectual property questino more vague, but it has still the same answer to the core.

  • Finnn
    Offline / Send Message
    Finnn greentooth

    I disagree. Why would it not be important to think about ethics? Especially with the level of disruption you describe, it becomes even more important to discuss ethics or legality. I never said that Id like to file a lawsuit. I am a subscriber of midjourney myself and I enjoy their tool, but imo a discord about the ethics is as important as discussing the future of art.

  • ZacD
    Offline / Send Message
    ZacD sublime tool

    I should clarify, the ethics relating to artists works being used to feed the model I can't bring myself to care about. Purposefully ripping off or imitating art and profiting off of it is still in a similar place as it was before Dall-e, it's just a little bit easier. There are plenty of other ethical issues around AI generated images, particularly biases.

    I'm also more concerned what this means for fake portfolios and art tests. And in the near future, what concept artist's role and workflows will be like in the next 5-10 years. And how quickly this is going to bleed into the 3d space.

  • Finnn
    Offline / Send Message
    Finnn greentooth

    What do you mean with biases?


    Fake portfolios could be a concern, but its also easy to filter out someone who fakes their portfolio through tests.

    And yes, I think concept artists are the ones that will have to adjust the most. Iteration speed will increase alot, which is not necessarily a bad thing. :)

  • Tiles
    Online / Send Message
    Tiles polycounter lvl 10

    It is not your artwork that is published. It is a AI had a look at your image. Then paints in your style. Watercolors for example. Same could have been me, looking at your picture, then use your style. Your content, and everything that you can claim copyright for, is never touched. What you ask for here is that nobody is allowed to use watercolor anymore since this and that artist has used watercolors for his image.

    I don't worry for the artists too much though. 99% of the output is plain garbage as i have already found out. And the rest is also most of the times not what you are looking for. It is yet another tool in the pipeline that can speed things up. But not necessary.

  • Finnn
    Offline / Send Message
    Finnn greentooth

    You are missing the point and misunderstood my last comment entirely. I didnt say that art generated by the AI and published should not belong to the one creating it using the AI. I am talking about the dataset and the use of intellectual property without consent.

    Again, you are saying content on Artstation is "free content", thats what I was referring to.


    And when you say 99% of the output is garbage then you are blind: I just screenshotted the community feed for you. Does this look like garbage to you?


  • Tiles
    Online / Send Message
    Tiles polycounter lvl 10

    Again, you are saying content on Artstation is "free content", thats what I was referring to.

    No, i did not say this, and here you misunderstand me. Free would mean that you could grab the images at Artstation and use it for your own needs, put it into videos, make your own images with it ant so on. Like CC0 content. But the AI does not break any copyright by just looking at your images and analyzing it. As told, the same does every other artist that visits Artstation. Or by visiting the Louvre and studying the old masters. Or by simply looking out of the window. (Crazy that you are not allowed to make a picture of some buildings anymore)

    And when you say 99% of the output is garbage then you are blind: I just screenshotted the community feed for you. Does this look like garbage to you?

    I talk about personal experience. I try to use SD images at the moment to make a video. What you show is the 1% of useful content after lots of refinement steps. But from this 1% it's again just a fraction close to what you wanted to achieve. It's throwing a dice. There is not this much control over the result. The same terms will give you always a different image.

    Yesterday i have created around 1000 images, and just a handful looks not completely destroyed. And i have yet to find the image that looks exactly how i wanted it to be.

    This is the average result when you try to create a happy woman in a red dress. The 99% garbage. Midjourney is a bit better. But not this much. As told, i don't worry about artists at this point ;)






  • Finnn
    Offline / Send Message
    Finnn greentooth

    Saying that 99% of the output is garbage makes no sense.

    It is a work in progress and its part of the AI generation process to reroll and iterate & prompt craft to get the results you want. It takes time and iterations. Just like any other process creating art does.

    Its the same with sculpting.

    Does the image below look like an epic dragon skull? no. because its not done yet. Same for the images you sent.

    So yes, the screenshot of the community feed is representitive of the quality that midjourney can produce. You dont measure ZBrush's performance as an art tool through the visual quality of work in progress sculpts, do you?


  • Tiles
    Online / Send Message
    Tiles polycounter lvl 10


    Well, I already provided proof for my experience :)

    Most of the times there is no further refinement possible. The prompt allows just so and so much keywords. And the AI has its own mindset. I agree that with the right keywords you can get good looking and even awesome results. But most probably completely different from what you want to achieve. It's like, hey, i wanted to create a space station, but this cool cybercity is much better.

    Try to create a girl face with wrinkles under the brown eyes, the flowers in her hair in another color, the eye brows a bit higher than in the former iteration, and so on. The things that you call refinement. It will not refine, the image will change completely. It's not that it grabs the existing face and adds the missing bits. Not even with image to image ( this requires a good looking base image already, so you are again trapped) . And looking to the left or looking to the right is something that the AI completely ignores most of the times. I had a hard time to let a human at least face the camera. I always ended in a back view.

    Working with the AI solutions is not like modeling and refining until you are happy. You throw a dice. And hope that a useful result is included. And then you throw another dice. And another one. And as told, here 99 of 100 images is simply garbage. Maybe we can say, the results are still fine, but useless for what you wanted to achieve.

    Your dragon, finished or not, is at least to recognize as a dragon at any step. While with SD, Midjourney and co you get 99 images of 100 that doesn't even look close to a dragon. I know what i talk about, i also had my fun with dragon images already.

  • Finnn
    Offline / Send Message
    Finnn greentooth

    Did you consider that you just dont know (yet) how to use the AI properly?

    There is a random component, yes. But you can very well prompt craft and refine until you get the results you want. I was able to combine various elements that I wanted to see. The more specific and the more elements you want, the more difficult it will be, but its not impossible to get exactly what you are looking for. It obviously has less control over the result than a traditional digital artist.

    But your assumption is just wrong: You provide proof for your own methods result in 99% bad images, but how does that transfer to everyone that uses AI to generate images and to people who have practiced using the AI?

    The quality and "likeliness of closeness to your idea" of your artworks mostly based on your skill of prompt crafting:

    + using the right words

    + using the right combinations (style combinations, word combinations)

    + considering prompt order

    + text weights

    + image weights (style transfer)

    + right ratio for the idea (portraits work better in --ar 2:3 for example)


    I recommend you to check out this extensive collection of keywords that will help you to not get 99% bad results, trust me. :)

    https://github.com/willwulfken/MidJourney-Styles-and-Keywords-Reference/blob/main/Pages/MJ_V3/Style_Pages/Colors_and_Palettes.md


  • Tiles
    Online / Send Message
    Tiles polycounter lvl 10

    Did you consider that you just dont know (yet) how to use the AI properly?

    :D

    There is a random component, yes. ... It obviously has less control over the result than a traditional digital artist.

    That's what i talk about. The general concept behind AI art. Not if i am just too dumb for the job :)

    It's not you paint the rough structures and then refine. It's throwing a dice. With every throw you might get a bit closer. But it can also be the exact opposite. That you move away from what you want to achieve.

    I recommend you to check out this extensive collection of keywords that will help you to not get 99% bad results, trust me. :)

    Not to forget this one, which i use in conjunction with Stable Diffusion: https://lexica.art/?q=drum

    Which should answer your question if i know how to use it. I get my results.

    But again, i define a bad result when it is not what i am after. And of course the really bad results when the AI simply doesn't want to do what i want it to do. Everything around humans is still problematic. Just search through the results at lexica.

    And some things are simply not to achieve. Ladybugs for example ... https://lexica.art/?q=ladybug

  • Iwazaruk7
    Offline / Send Message
    Iwazaruk7 triangle

    Deviantart, artstation and others should allow filtering AI-made works as option, to prevent mass confusion.

    Perhaps even make a separate subsite for that.

  • Tiles
    Online / Send Message
    Tiles polycounter lvl 10

    Some platforms like Getty forbids AI art already ...

    https://twitter.com/mediagazer/status/1572596647774887936

Sign In or Register to comment.