Working On New Modelling Software
Hello everyone,
I’m an AI engineer and software developer. I’ll cut to the chase because I know your time is precious. Game development is hard. I don’t need to tell any of you that. My goal is to make game development (CG art) easier for everybody. I’ve been working on a suite of tools to do just that.
The first tool is an AI modelling assistant. Where you literally feed it a single image and it will attempt to model the subject in the image for you. Parts of the image that are not present (say, the interior or backside of a car if that’s what you were trying to model) will be creatively designed by the AI while trying to maintain visual coherency. The goal is to provide an artist with either a starting point for their project or if they short on time, a finished model.
Another motivation that I had going into this was the high cost of 3D modelling and visualization software (thank God for Blender!). With subscription models a norm these days, its getting harder to outright own a license for the tools you need. Which drives me crazy. I suppose there could be some benefits for low entry cost and temporary use of software, but I personally have never been a fan of that business model.
So all of that to say this; I want to provide artists and developers with powerful modelling automation at a low cost with a lifetime license.
Why do I want to do this? Well, because I know how frustrating it can be to have a story to tell, or something you want to express visually but you lack the skills or resources to do it.
As of right now, it works well with a mix of hard-surface, mechanical type shapes and naturally occurring shapes from nature such as rock structures and various vegetation.
My long term goal is to fuse in knowledge of human and animal anatomy as well. Although that will take some time.
Anyway, the software is still in-work and I'm not really at a point where I can show off results just yet. I expect to have some results I can share in 3 weeks or so. In the mean time I was pretty excited and just wanted to share it with the community.
Replies
In my humble opinion 3d modelling soft itself is quite ok now. Easy to use and quick except certain complex scenarios , those need to be procedural and adjustable on the fly . If Ai could suggest node flow . a math formula to accomplish certain thing it would be nice indeed.
Texture soft on the other hand is monstrously inconvenient , underdeveloped . lacks simple basic conveniences . Even SPainter while being a bit better still lacks so much and you have to basically re-do everything. And it's pain in your a.. to create custom things. All the AI stuff I saw yet in texture field is so hopelessly useless unfortunately and leave a huge empty space to fulfill.
Well ... if anything it would be great for some of the sketch-based AI modeling software shown in research papers over the years to finally be made available to the public.
Besides that, while getting an AI to gererate a model from photos is always fun, being able to generate accurate models from 2d lineart would be the actual game changer imho. For instance : sketching a shape, getting it roughly recognized as a first pass blob, and then providing more sketches on top of scrennshots of said blob to refine things it further - rince and repeat.
This would essentially be modeling but without having to constantly interact with a viewort and without ever having to pull/push verts in space. Some apps do have a little bit of that (Curvy3d and 3DCoat) but only to a very limited extent, like with the 3D coat silhouette tool.
The closest I could ever find to such a workflow in Blender consists of sketching the main lines of a design as 2D curves, arranging them manually in space to form a loose wireframe, and then later filling things in. But while it certainly works in some cases it is less direct that potentially being able to start from a 3/4 perspective sketch and refining from there.
Those are all great ideas.
Texturing is something I'd like to tackle in the future, although I'm not quite sure how I'd do it now. The number of things I have textured amount to 0 so it will require a lot of learning. The pain point of any AI project is gathering/creating data. Creating art is a slow process so gathering that data can take a long time...but researchers are pretty clever these days in how they fill those gaps so we shall see.
As for the 2D line art based modelling. that is definitely something that can be done and will likely be an upgrade in a future release. Of course I'm getting ahead of myself because I'm not even done with the first version. Ultimately it would be good to cater to different skill levels. Photos are great for people who have less artistic ability and just want to make something in a single shot (myself, and most of my programming friends). This approach has little to no control over the finer aspects of design. Of course an artist wants finer control over there work and still may want to enjoy some benefits of automation a long the way. Having both design options at your disposal will be ideal I think.
Unless of course Epic Games does all of this first. Which would be awesome. There is Metahuman though, which is amazing. I'm not sure if they use AI, or predefined models and blendshapes, but still amazing what can be done these days.
Hello
in my humble opinion and very little knowledge, there are plenty of photogrammetry tools. cleaning and retopology take time to make it useable but they are awesome for many use-cases. I'll love to see your tool in action hopefully it makes awesome AI-created models
I like to use software that can solve pipeline problems and make workflow fast, I have to use a bunch of software to make a model for that I have to learn very complex tools. if your software can replace some of the tools from the pipeline and make workflow fast everyone likes to use it.