Testing Usability of an Xbox Kinect to aid with Game art

Defaultsound · Nov 2013

So currently at University I am working on a 3D project regarding the usability of an Xbox Kinect and whether or not it has potential in aiding with the workflow of creating assets.

What triggered this idea is that 3D scanning has become and is becoming more and more predominant in game studios, however the solutions are fairly high cost (Think Infinite Realities). Now Photogrammetry is a low cost solution that has proven to be possible to use in an asset creation workflow for example Rustclad an indie game in development explained this was possible. Yet the Kinect could possibly hold value in being a tool artists could use, given its low cost (low as £25) it would make sense to explore its potentials and rule out whether or not it would be useful or not.

So currently I have narrowed it down to three experiments. These experiments will attempt to see if the Kinect holds any value in aiding with the production of game assets,

1. Scanning a Hard surface (Baking AO and Normals)
2. Scanning Cloth (Baking AO and Normals)
3. Scanning entire object (Baking AO and Normals)

Now there are pit falls in these experiments which I will discuss later but for now I wish to gain qualitative feedback from you all regarding the outcome of these tests. Merely in the form of a debate and seeing whether or not the likes of the Kinect could be used in the lights of Freelance and Indie Developers

So a survey for the first test.

The first test was merely to see if a hard surface could cleanly be scanned and baked for use in engine. Hypothesis was that the Kinect would bake the maps and possibly speed things up a bit.

Here was the result:

[ame="

http://www.youtube.com/watch?v=Z2mEhbW6jfQ"]Hard Surface Kinect Scan, Experiment One - YouTube[/ame]

A Rendered:

[SKETCHFAB]cc7ac093d5b44ff7b35fd66e4967803f[/SKETCHFAB]

So from this it was clear that it could scan the details fairly well even given the limitations of the Kinect.

So I ask. (To spark some debate)

Quality wise, would you consider using this?
Would the tested workflow be of any use or interest to you?
Are we going to see more 3D scanning in the future, why or why not?

Thanks and enjoy

marks · Nov 2013

Defaultsound wrote: »

for now I wish to gain qualitative feedback from you all regarding the outcome of these tests.

What do you actually want to know? One looks noisy and "scanned". The other looks clean and as if it was authored in a DCC app.

Jason Young · Nov 2013

B looks better.

Steppenwolf · Nov 2013

Shouldn't you first try to get some decent results out of the Kinect scan to begin win? I mean it's no rocket science that A is useless in that form. Don't need a survey for that.

Would make more sense to get it to a point where the visual quality is comparable and then take a look at which method is less time consuming.

ZacD · Nov 2013

Also it'd be better to pick assets that play to the strength of the Kinect more. A flat surface with easily tiled details isn't going to be very favorable to the Kinect. Pic an object that does not have small details and isn't easily modeled or tiling.

Luke_Starkie · Nov 2013

Scan some faces, that seems to be done a lot these days, see how good you can get the results with the kinect.

Defaultsound · Nov 2013

OK guys thanks for the responses, however from asking another group people it has dawned on me that this way of exploring options for the Kinect was the wrong way. (Pretty silly of me but I was stuck with how I could test the usability.)

So Instead I have devise this as I quote:

So I've had a think and decided that this is the wrong way to explore the usability of the Kinect. it dawned on me that this comparative data wouldn't really be useful in anyway. So that said I have worked out a better solution. Later today I will upload a video discussing the potential of scanning meshes with a Kinect. The video itself will be a realtime demo of the workflow I used to create A (Essentially its still an experiment but less about comparing and more on exploring workflows and seeing what works and what doesn't) however I will discuss other areas this scanned data could be used as. On top of that I will open this up for discussion to gain a basic view of everyone's opinions to see whether or not it is something worthwhile in the communities perspective. From these basic experiments the flaws can be ironed out and given that I will share my findings leave it up to others to see what they can devise if they wish to try experimenting with a Kinect to aid with their game art. Ideally I will try and solve the basic problems and given this share this with others.

Shouldn't you first try to get some decent results out of the Kinect scan to begin win? I mean it's no rocket science that A is useless in that form. Don't need a survey for that.

Would make more sense to get it to a point where the visual quality is comparable and then take a look at which method is less time consuming.

Yes you are correct, however I will instead of trying to achieve the best results possible it will begin with just attempting to devise a means to use scanned data, then perhaps taking this further to get better results.

Also it'd be better to pick assets that play to the strength of the Kinect more. A flat surface with easily tiled details isn't going to be very favorable to the Kinect. Pic an object that does not have small details and isn't easily modeled or tiling.

Exactly right, basically with all experiments I knew that a flat surface wouldn't be an idea situation, however given the nature of the test I wanted to rule it out and see how far it could be taken.

Also I should expand the reason for exploring the Kinect as a tool which I shall, I will be updating the OP as I speak.

EDIT:

Video of the first test:

[ame="

http://www.youtube.com/watch?v=Z2mEhbW6jfQ"]Hard Surface Kinect Scan, Experiment One - YouTube[/ame]

Defaultsound · Nov 2013

Update:

Experiment two has started. So far I have scanned cloth and made a usable high poly for baking. Part two is work in progress.

[ame="

http://www.youtube.com/watch?v=4kC695WJ7xc"]Scanning Cloth with Kinect for Games, Experiment Two Part 1 - YouTube[/ame]

Also I don't know if a moderator could perhaps move this thread to P&P since it may be more applicable there.

Defaultsound · Nov 2013

And the final results of experiment two are up.

[ame="

http://www.youtube.com/watch?v=DcRnG57bla4&"]Scanning Cloth with Kinect for Games, Experiment Two Part 2 - YouTube[/ame]

Overall really happy with the outcome of this test. I could see the Kinect having a lot of potential with assisting with game art. More to come.

AlecMoody · Nov 2013

I have had better luck scanning with 123d catch than with the kinect. If you need object scale, you have to include a reference. For the best results you need to prep the surface, add reference points, and control the lighting.

I haven't been using it for game work though- Too much cleanup work to get a high quality model.

artquest · Nov 2013

So I know your project isn't really about this but I would like to propose a slightly different angle for the kinetic technology that I think might be more interesting for future game content creation. Finding more methods to scan data is I think a pretty cool idea, however I think the cost problem will eventually fix itself. So to spark discussion I'll post the idea here.

As someone who has severe aches and pains from sitting at a desk all day creating art. I would love to be able to use the kinect technology for the computer, freeing artists from having to use the mouse and keyboard so much. If we could grab a vertices by pinching the air and moving our hands, we could model standing up and avoid so many of the health problems that come from too much sitting.

The other great thing is that because we are in 3d space already, grabbing verts in the air and moving our hands is so much more intuitive for 3d modeling instead of dealing with a widget that only moves in two dimensions at a time or less.

The kinect technology offers so much in the way of future device input. I hope I live to see the day when it completely replaces the mouse and keyboard

fearian · Nov 2013

Hey dude, 3d scanning technician here - I can't say much about the kinect (never used it!) but for my 2 cents, much like Alec Moody said, photogrammetry software such as 123d catch or Agisoft an so on would be much more useful for gamer art. The biggest reason for me is texure quality. With camera based solutions you will end up with a mesh that has very accurate, very high quality colour data with it. You can use this to create displacement maps that will provide an extra layer of detail to the model. and if you where studio funded and had invested in a polarised light set up, you would have a huge headstart on diffuse textures as well.

Clean up is always an issue with 3d scanning, but will a well developed system you can minimise it hugely. In Games, scanning is one of those approaches that is very context specific. For cloth, you would be better off running a simulation and baking it down!

Defaultsound · Nov 2013

AlecMoody wrote: »

I have had better luck scanning with 123d catch than with the kinect. If you need object scale, you have to include a reference. For the best results you need to prep the surface, add reference points, and control the lighting.

I haven't been using it for game work though- Too much cleanup work to get a high quality model.

Ok thanks for the info. I know that the likes of photogrammetry may have far better results than the Kinect. However the idea of this project is to rule out whether or not the Kinect has potential and seeing what applications it could hold if it does. At the moment it has proven worthy but further study would have to be taken out to compare it with say the likes of Photogrammetry. Also Rustclad have tried using 123d catch to produce art assets and it worked quite well.

So I know your project isn't really about this but I would like to propose a slightly different angle for the kinetic technology that I think might be more interesting for future game content creation. Finding more methods to scan data is I think a pretty cool idea, however I think the cost problem will eventually fix itself. So to spark discussion I'll post the idea here.

As someone who has severe aches and pains from sitting at a desk all day creating art. I would love to be able to use the kinect technology for the computer, freeing artists from having to use the mouse and keyboard so much. If we could grab a vertices by pinching the air and moving our hands, we could model standing up and avoid so many of the health problems that come from too much sitting.

The other great thing is that because we are in 3d space already, grabbing verts in the air and moving our hands is so much more intuitive for 3d modeling instead of dealing with a widget that only moves in two dimensions at a time or less.

The kinect technology offers so much in the way of future device input. I hope I live to see the day when it completely replaces the mouse and keyboard

Interesting idea. I know that Xspace had an example of this with the Leap Motion, again could be interesting to look into for another project. (I'll keep this in mind)

Hey dude, 3d scanning technician here - I can't say much about the kinect (never used it!) but for my 2 cents, much like Alec Moody said, photogrammetry software such as 123d catch or Agisoft an so on would be much more useful for gamer art. The biggest reason for me is texure quality. With camera based solutions you will end up with a mesh that has very accurate, very high quality colour data with it. You can use this to create displacement maps that will provide an extra layer of detail to the model. and if you where studio funded and had invested in a polarised light set up, you would have a huge headstart on diffuse textures as well.

Clean up is always an issue with 3d scanning, but will a well developed system you can minimise it hugely. In Games, scanning is one of those approaches that is very context specific. For cloth, you would be better off running a simulation and baking it down!

Again I know of photogrammetry applications, however specifically looking at the Kinect for this project. For simulating cloth I had that in mind and I was going to compare the two so again this would need further study. One question though about using photographs. Would the like of say using a single camera give similar results to using a multi camera set up? I would assume there could be inconsistencies between the two.

Defaultsound · Nov 2013

[ame="

http://www.youtube.com/watch?v=Gm_r2QtbVyI"]Scanning Objects with Kinect for Games, Experiment Three - YouTube[/ame]

Last experiment and concluding my findings. So overall I think the Kinect may not hold valuable necessities as a tool for artists. However it could be tested further and for some it may be useful for learning and understanding the proportions of geometry perhaps.

While this research may seem a no brainer and seem quite obvious to what to expect, I still wanted to carry out the tests and do a proper write up regarding the potential of the Kinect. What works what doesn't and where it could be used. When I do get round to writing a dissertation I think photogrammetry will be the focus of that in relation to application asset creation. Given everyone's thoughts so far.

AlecMoody · Nov 2013

imho scanning volumes has limited usefulness as a process for game artists. I have worked on games where it was critical part of the workflow but the applications are narrow. You need to be developing a game where the objects to be scanned are something you can actually get physical access to. That rules out many games that use concept art not based on real life objects. Also, I have found that scan data helps dial in model accuracy but it does not necessarily save a lot of time. Also, at best it is very accurate reference material or possibly something to be dynameshed and heavily worked.
For material capture it is awesome. I buy lots of scans off of surface mimic and use them for micro texture passes on my sculpts.

Defaultsound · Nov 2013

AlecMoody wrote: »

imho scanning volumes has limited usefulness as a process for game artists. I have worked on games where it was critical part of the workflow but the applications are narrow. You need to be developing a game where the objects to be scanned are something you can actually get physical access to. That rules out many games that use concept art not based on real life objects. Also, I have found that scan data helps dial in model accuracy but it does not necessarily save a lot of time. Also, at best it is very accurate reference material or possibly something to be dynameshed and heavily worked.
For material capture it is awesome. I buy lots of scans off of surface mimic and use them for micro texture passes on my sculpts.

Thanks for that info, very helpful. As for texture scanning that might be something else to consider. I know after reading about the scanning system they are using for Order 1886 it looks very cool.

Testing Usability of an Xbox Kinect to aid with Game art

Replies