Home Career & Education

Sloppy work or poor directions - Sony presentation on how much they cost and how to measure it

This presentation shows how key performance indexes are used at Sony to manage quality and costs of outsourced work.  If you're a sloppy artist (unwelded verts, uneven mesh density, bad uvs, etc.) then this will give you some concrete numbers on how much your sloppy work costs.  It also works the other way by showing the costs of unclear direction and management.
https://youtu.be/X23-4CAt7ic

Replies

  • Bruno Afonseca
    Interesting perspective, thanks for sharing it :) Those little problems surely create a lot of unnecessary back and forth, and I believe it's worth investing some time in the artists to teach them ways to minimize it.
  • JEmerson
    Offline / Send Message
    JEmerson polycounter lvl 7
    This is a very relevant topic for studios that either outsource work and/or studios that do outsourcing. KPIs are an imperative, and the methods that Mr. Otake present are easy to understand and relevant. While I do have an issue with some of the presentation (particularly the review phase elimination due to the lack of defining a standard of performance outside of simply reducing these), overall it was a very good presentation and articulately explains the usage of data science for firms and artists (even freelancers could do this for themselves using a simple excel table). Thanks for sharing this RyanB.
  • RyanB
    Glad you found it valuable.  I've spent a lot of time cleaning up other artists models, mostly for lighting, and I would see all of the errors described in this video.  I'd like to see this on any big project, not just a heavily outsourced project. 

    I've been watching presentations from outsourcing and HR expos and it's very informative.  I think we can learn a lot about how we artists are perceived and valued from these expos, for example Linkedin Talent Solutions (also called Linkedin Talent Connect) gave me some real insight into how every action we take is monitored, packaged and sold. 

    Artists starting out today hoping to get a job will be put through automated systems and if you aren't aware of how those systems work you will be at a disadvantage.



  • Mehran Khan
    Offline / Send Message
    Mehran Khan polycounter lvl 10
    thanks for this man, this really helps
  • Kwramm
    Offline / Send Message
    Kwramm interpolator
    XDS is really a great source of know-how how to better work with your outsourcer. As outsourcer, I can just say that it's incredibly important that clients have a good idea what constitutes quality, and that they communicate it to us. Even better: share your quality checking tools and checklists with the outsourcer.
    Not only will this make the client's artists happier, it will also save many costly feedback loops - think of the time wasted on skype and email. Think of the time wasted due to time difference. Think of the increased "lost in translation" potential that exists in every feedback loop. The fewer feedback loops the better - give your outsourcer all the info they need to get those assets right at their first shot!
  • rogelio
    Offline / Send Message
    rogelio greentooth
    Very interesting subject matter.  I agree most of the issues in the back and forth are generic issues.  At ND we outsource many assets and yes having to review assets on a daily basis for just our individual levels is time consuming.  That is one part of the subject that was not really shown here is that the review process goes both ways,  at least in every studio I have been in... the artists are responsible with making sure the assets are done correctly for the area in the game.  I think many artists wanting to get into the industry and especially AAA companies need to understand that you will deal with making sure assets are being sent out with instructions that are understandable for outsourcing studios, and putting these packages together can be annoying for artists that just want to art...  I find that most of the issues that come up during the outsourcing process for assets are due to bad instructions.  The opposite is true also when it comes to instructions being super detailed and complex.  Documentation helps also, but too much documentation can lead to further confusion.  Giving outsourcing companies bibles of how to art is not the right way.  The best route is having a Lead or Senior artists teach the outsourcing team either by bringing them in for a week or two or flying to the studio.   To me outsourcing is part of the team, and personally find it a way to delegate things I do not want to do at times.  If I have the look I want and I want to propagate it all over... outsourcing is perfect for that.  My job after all is to make the best possible art regardless of how I get to that.  The days that one artist does everything are gone.   As you work with studios you learn the strength also.  Some are amazing at hard surface and others are amazing at sculptural naturalistic stuff.  Having an understanding of the strengths of each studio will lead to better assets.  I happen to know a bit of the outsourcing pipe because I was part of the helping hand for substance designer workflows side of things also.  That was hard to get studios to adopt a whole new tool at the time.  Now it has become a standard for most studios so it makes it easier.   In the start assets would take longer than before because of the learning curve... we were hoping that with more automation within the substance tool sets that eventually productivity curve would get better, and eventually it did.  The quality bar is pretty standard now which is good.  We do not really get an asset that is just horrible anymore.  Everything has hit a good bar and we are just left with pushing good to amazing... lol.
  • Kwramm
    Offline / Send Message
    Kwramm interpolator
    rogelio said:
    Very interesting subject matter.  I agree most of the issues in the back and forth are generic issues.  At ND we outsource many assets and yes having to review assets on a daily basis for just our individual levels is time consuming.  That is one part of the subject that was not really shown here is that the review process goes both ways,  at least in every studio I have been in... the artists are responsible with making sure the assets are done correctly for the area in the game.  I think many artists wanting to get into the industry and especially AAA companies need to understand that you will deal with making sure assets are being sent out with instructions that are understandable for outsourcing studios, and putting these packages together can be annoying for artists that just want to art...  I find that most of the issues that come up during the outsourcing process for assets are due to bad instructions.  The opposite is true also when it comes to instructions being super detailed and complex.  Documentation helps also, but too much documentation can lead to further confusion.  Giving outsourcing companies bibles of how to art is not the right way.  The best route is having a Lead or Senior artists teach the outsourcing team either by bringing them in for a week or two or flying to the studio.   To me outsourcing is part of the team, and personally find it a way to delegate things I do not want to do at times.  If I have the look I want and I want to propagate it all over... outsourcing is perfect for that.  My job after all is to make the best possible art regardless of how I get to that.  The days that one artist does everything are gone.   As you work with studios you learn the strength also.  Some are amazing at hard surface and others are amazing at sculptural naturalistic stuff.  Having an understanding of the strengths of each studio will lead to better assets.  I happen to know a bit of the outsourcing pipe because I was part of the helping hand for substance designer workflows side of things also.  That was hard to get studios to adopt a whole new tool at the time.  Now it has become a standard for most studios so it makes it easier.   In the start assets would take longer than before because of the learning curve... we were hoping that with more automation within the substance tool sets that eventually productivity curve would get better, and eventually it did.  The quality bar is pretty standard now which is good.  We do not really get an asset that is just horrible anymore.  Everything has hit a good bar and we are just left with pushing good to amazing... lol.
    First a big thanks to the guys at ND, because working with you is always a pleasure :)

    Great points - finding the right balance of documentation. Not every studio can afford to hand-pick documentation for the outsourcer, but it does cost us time when we have to dig through tons of docs just to find out what concerns us and what doesn't. Outdated docs are also a major annoyance for us. Us figuring out YOUR pipeline wastes YOUR money - because we could just use that time to get art done.

    When dealing with outsourcers in different cultures, language is also very important. Slang, colloquialisms, complex language, abbreviations can all cause confusion and can be difficult to communicate. Another important thing is to clarify how literally your outsourcer should take concept art and what level of freedom they have in regards to interpretation of instructions and concepts. This should be clarified beforehand. Some clients want to double-confirm everything, others totally trust us, others don't clarify yet complain after all the work is done (please don't be that client!).

    Another good thing to keep in mind is quality vs. speed; Basically: quality, price, speed - pick 2. It's important to communicate to your outsourcer where you can compromise and where you can't. Of course we aim to please in all 3 areas, but practically there is often one that comes up short unless you have a tip top super perfect pipeline. E.g. quality for unimportant props may be different from quality for hero assets. Tech accuracy may be, to some clients, less important, for certain assets than for others. We can also see this in the checklists clients give us - some clients are quite relaxed, yet others are super strict and bounce back assets due to minor things. If it's not obvious, communicate it to your outsourcer to avoid time wasted in feedback loops.




  • Justo
    Offline / Send Message
    Justo polycounter
    I work in an outsourcer and can totally relate to this stuff, thanks for sharing Ryan. 
  • rogelio
    Offline / Send Message
    rogelio greentooth
    Funny how we can be so excited about talking about this subject :)
  • RyanB
    rogelio said:
    Funny how we can be so excited about talking about this subject :)
    I'm a weirdo who actually enjoys optimization and making sure everything is clean.  :)  It makes everything better.


  • Polygoblin
    Offline / Send Message
    Polygoblin polycounter
    really informative, thx for sharing! big fan of tracking my own data for efficiency, so more methods to assure the best value for everyone's time from the outsource perspective is greatly appreciated.

    the overage from generic issues in a bar graph is quite the eye opener. 
  • JordanN
    Offline / Send Message
    JordanN interpolator
    I haven't watched the whole video yet but I do have a question. Why would a studio accept outsourced work that is [consistently] sloppy? Or better yet, why not avoid the headache and just hire a full-time/part-time artist instead? 

    When I read the header in the OP, my eyesbrows lit up seeing this  "unwelded verts, uneven mesh density, bad uvs, etc.", because I thought those were all fundamentals 3D artists were taught to avoid.
     
  • Kwramm
    Offline / Send Message
    Kwramm interpolator
    JordanN said:
    I haven't watched the whole video yet but I do have a question. Why would a studio accept outsourced work that is [consistently] sloppy? Or better yet, why not avoid the headache and just hire a full-time/part-time artist instead? 

    When I read the header in the OP, my eyesbrows lit up seeing this  "unwelded verts, uneven mesh density, bad uvs, etc.", because I thought those were all fundamentals 3D artists were taught to avoid.
     
    Hiring 30 to 60 AAA artists on short notice and making them work as a team from day one isn't that easy. There are set plans when the team will have to ramp up and ramp down and if this doesn't happen then schedules slip and the client's project has a problem. You don't want to pay those guys freelance rates either, because then it becomes quite expensive. Then add costs for software licenses and hardware on top. Many outsource projects are also quite short. e.g. 3 - 12 months max. Too short for many people, who would rather be looking for a permanent job.

    As a studio, you can forget all those hassles when you work with an outsourcer. They have a pool of artists or teams to pick from at negotiatable package prices (per asset, per map, per week, etc.)  and there is no issue with hiring or firing because everyone is already on-board. And if your game needs DLC or an expansion or a part II, 6 months later, then you just come back to your outsourcer and say that you need the same team again for more work - they already know the drill.

    Basically the outsourcer doesn't just do art - you also outsource training, team-building, team management and hardware/software setup. And you don't have to fire entire teams at the end of the project (like in gig based VFX production). At they outsourcer they will just work on something different after your project is finished. Especially with projects from big studios who share tech and engines a lot of experience carries over from project to project.

  • Sk0LLiE
    Offline / Send Message
    Sk0LLiE polycounter
    Interesting stuff, cheers!
Sign In or Register to comment.