Your Art Test is BAD, this is why (And how to improve it)
greentooth
Hi, its been a while.
A couple days ago this article was posted on 80.lv and this in a way is a response to it:
https://80.lv/articles/game-art-tests-and-how-to-approach-them/
TL DR:
1. Select carefully who you are sending tests to, if the applicant is way below the required skill level there is no point to a Test
2. Make your Brief as detailed and specific as possible, look into ways to share workflow documentation with the applicant, censor sensible information in the documentation.
3. Allow artists to show atleast screenshots of the finished result
4. Give Feebdack
5. Instead of giving a Test consider giving Portfolio tips and tell them to reapply in the future.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Among all the toxic employment practices in the industry Art tests are definetly up there with the worst IMO, let me explain why.
Now preemtively let me say not all Art tests are bad, just roughly 80-90% if your tests belongs to the other 10-20% congrats on being nice
Now in order to line out why most Art tests are bad we have to clarify the purpose of an Art test:
Issue NR. 1
This is a probably the biggest Issue with the whole system, hence the first place.
Some companies automatically send out Art tests to all applicants of certain openings, now this is just deliberately wasting everyones time.
Chances are lower skilled applicants will not recognize their lacking skill to be even considered for a position.
But they will invest 2 weeks of their live into this test, burning themselves out, possibly learning absolutely nothing due to issues 2. 3. and 5. and when they send their test in, it either doesn't even get reviewed or the reviewers time is wasted since the candidate was no match which in probably 90% of cases could have been avoided by spending 1 minute to check the applicants portfolio first.
You can ignore applicants, reject them immideately, whatever, nothing beats this. If your company is doing this and you are responsible I urge you to explain yourself because I fail to think of any other reasons then lack of respect for other peoples time or straight up malice. (Perhaps laziness but in the end this will actually occupy more of the reviewers time, so that doesn't make too much sense)
Issue NR. 2
Most art tests have 1-2 pages of Briefings and absolutely no workflow documentation. Now lets put this into perspective, most companies have dozens if not hundreds of pages of internal workflow documentations for their employees to rely upon.
Now lets get this straight: you expect someone who has not worked for your company before to produce a result to the same specifications as your employees, who are already experienced and trained in your workflow, without any guidence? And you think that will select the best candidate for the Job? Sure some of the candidates will get lucky and produce something that is vaguely like what you expected. But can you be certain that some of the other candidates wouldn't have produced a better result given the right instructions?
Now let me adress the most likely counter points to this argument:
But my Documentation is a company secret!
We'll thats what you make people sign NDAs for isn't it? If its too revealing you censor some sensitive parts, also if this is not your first game the documentation will largely be based on previous released titles. Most workflows will not be unique to your studio yet being aware of them will help applicants deliver what you expect to see in their work.
Or perhaps you are actually part of the 10-20% good tests. Its hard to falsify the possibility of a better candidate for the opening.
But if you ever had a candidate you hired after he passed the test who didn't pass probation or you missed your Growth Projection Target, then you have no one to blame but your test and yourself.
If you have ever switched companies, and since you are a hiring manager and have been around for a while you probably have, you should be aware that every company has sometimes slightly, sometimes drastically different workflows.
So applicants will work with the constraints they know since you are not giving them any documentation and those can be vastly different from whatever you are expecting.
To you an N-gon might be the gravest offense you have ever layed eyes upon, the environment artist might have 5 years of experience working in UE4 and is used to it triangulating on import, while your Engine might crash, erase the subversion history, set the servers on fire and burn down the building as soon as it gets the faintest smell of a pentagonal shape.
Issue NR. 3
Alright so you are telling me you think a candidate who spent months or years building their Portfolio is going to produce something in 2 weeks that is going to change your view of his abilities?
I think that is highly optimistic.
Additionaly you are not only judging a persons artistic skill, but also their ability to carve out time form their lives. Is that really a quality you are looking for in your artists?
Someone who is working his 9-5 to make a living will be at a massive dissadvantage compared to a freelancer who can just take off work for 2 weeks whenever he wants to apply for a job. And if you think someone should take their 10 days of annual leave off to burn themselfes out for your test, for a chance to get a job, I truly have no words for you.
The usual justification for this point is:
What result do you get if you give an Artist a task but only half the usual time to complete it?
A Bad result.
Corners can be cut, shortcuts can be taken, but is he going to produce his best work? Hell no, but thats what you want to see isn't it?
And this produces another angle of uncertainty, cause ofcourse you don't say where to cut the corners, so Artist A might save time on the lowpoly while Artist B chooses to save time on the Highpoly or the Texture.
So wich one is more forgivalbe? a lacking technical execution or a lacking presentation? Probably neither, but the time has to come from somewhere.
Issue NR. 4
So you make people work 14 days for you for free and then they are not even allowed to show what they did.
There is a multitude of possible reasons for this that make this simultanously the easiest and hardest issue to fix.
Is it so that people can't "cheat" by looking at other peoples results?
That seems pretty weird, I can look at Michael Angelo's David, that doesn't mean I am able to sculpt it.
Is it because you are sending them a concept from your unanounced Game?
Well maybe use a concept that has already been released to the public.
Additionally you can put everything under NDA except the finished Screenshots, selected and authorized by you. That way future applicants don't know about technical restrictions and chances are they will have to redo most of their work for the test if it wasn't created using your technical specifications.
Issue NR. 5
So I have worked on perhaps roughly 20 different Games over the course of my Career, it has NEVER happened to me that the first asset produced for a new project was accepted without any feedback.
Thats right, you can give the most detailed specifications in the world, whatever you imagine the Applicant to produce, its basically impossible the result will be exactly like your expectation.
This is where the helpful process of giving feedback comes in handy, you might not get exactly what you wanted in the end, but atleast it will be closer to your expectations then whatever was delivered at first.
+ I imagine your daily routine does involve the application of feedback in some form, so it would be pretty handy to know how well an artist can implement direct feedback before hiring him?
What can I do to improve it then?
1. Screen your Applicants before sending out a test
Alright lastly I will present and counter some common arguments I am anticipating:
All of these suggestions take time, I don't have time
In the end you get what you give, the more time you put into making a better test the better the results you will get and chances are by having a fair testing practice you will also end up selecting more suitable applicants for the Job.
A lot of theese adjustments don't take much time, infact more carefully selecting who you give out tests to will help you save a lot of time reviewing tests and having more specific guidelines will help you find the right applicant faster, again saving you time in total applicants you need to review.
I am giving everyone a fair chance by giving a test to everyone.
Are you even looking at all the tests you get send? Is it faster to download a file, extract it, open it up in the software then just looking at a Website?
If your reply to the first Question is no, then how is that fair if you are not even reviewing everyones work?
The thruth is you are wasting your Applicants time and your time.
If you really are reviewing everyones tests I imagine thats not your favorite activity of the day and you'd rather be doing something else so saving time on that should be in your own interest.
I get 500 Applications a week I need to filter them somehow.
Again, is it faster to download a file, extract it, open it up in the software then just looking at a Website? Also if you get that many applicants why do you even need a Test? If its for some highly specific skill you are unable to judge from peoples Portfolios maybe that makes sense, in any other case it probably does not.
A couple days ago this article was posted on 80.lv and this in a way is a response to it:
https://80.lv/articles/game-art-tests-and-how-to-approach-them/
TL DR:
1. Select carefully who you are sending tests to, if the applicant is way below the required skill level there is no point to a Test2. Make your Brief as detailed and specific as possible, look into ways to share workflow documentation with the applicant, censor sensible information in the documentation.
3. Allow artists to show atleast screenshots of the finished result
4. Give Feebdack
5. Instead of giving a Test consider giving Portfolio tips and tell them to reapply in the future.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Among all the toxic employment practices in the industry Art tests are definetly up there with the worst IMO, let me explain why.
Now preemtively let me say not all Art tests are bad, just roughly 80-90% if your tests belongs to the other 10-20% congrats on being nice
Now in order to line out why most Art tests are bad we have to clarify the purpose of an Art test:
The purpose of an Art test is to help you select the best suited Applicant for your Open Position.
So lets figure out why most Art test are very badly suited to fulfil that prupose and what we can do to improve that.Issue NR. 1
Giving out Art Tests like candy
This is a probably the biggest Issue with the whole system, hence the first place.Some companies automatically send out Art tests to all applicants of certain openings, now this is just deliberately wasting everyones time.
Chances are lower skilled applicants will not recognize their lacking skill to be even considered for a position.
But they will invest 2 weeks of their live into this test, burning themselves out, possibly learning absolutely nothing due to issues 2. 3. and 5. and when they send their test in, it either doesn't even get reviewed or the reviewers time is wasted since the candidate was no match which in probably 90% of cases could have been avoided by spending 1 minute to check the applicants portfolio first.
You can ignore applicants, reject them immideately, whatever, nothing beats this. If your company is doing this and you are responsible I urge you to explain yourself because I fail to think of any other reasons then lack of respect for other peoples time or straight up malice. (Perhaps laziness but in the end this will actually occupy more of the reviewers time, so that doesn't make too much sense)
Issue NR. 2
Vague Briefings, no Documentation
Most art tests have 1-2 pages of Briefings and absolutely no workflow documentation. Now lets put this into perspective, most companies have dozens if not hundreds of pages of internal workflow documentations for their employees to rely upon.Now lets get this straight: you expect someone who has not worked for your company before to produce a result to the same specifications as your employees, who are already experienced and trained in your workflow, without any guidence? And you think that will select the best candidate for the Job? Sure some of the candidates will get lucky and produce something that is vaguely like what you expected. But can you be certain that some of the other candidates wouldn't have produced a better result given the right instructions?
Now let me adress the most likely counter points to this argument:
But my Documentation is a company secret!
We'll thats what you make people sign NDAs for isn't it? If its too revealing you censor some sensitive parts, also if this is not your first game the documentation will largely be based on previous released titles. Most workflows will not be unique to your studio yet being aware of them will help applicants deliver what you expect to see in their work.I don't have time to write a good Briefing.
Yes and perhaps that will teach you how valuable peoples time is. If you can't invest a day into compiling a proper test Briefing you will be the one hiring subpar applicants or failing to hire in a timely fashion.We got good candidates from our current test, why should I change it if it worked before?
Perhaps you got some good results, perhaps you don't care about other peoples time, perhaps you have years to find the right applicant.Or perhaps you are actually part of the 10-20% good tests. Its hard to falsify the possibility of a better candidate for the opening.
But if you ever had a candidate you hired after he passed the test who didn't pass probation or you missed your Growth Projection Target, then you have no one to blame but your test and yourself.
All I ask of my applicants is to use common industry workflows and deliver work that holds up to industry standarts.
This is a funny thing I noticed over the years, most companies think their workflow is THE industry standart and constantly are baffled by weird practices of applicants with industry experience, who should have known better.If you have ever switched companies, and since you are a hiring manager and have been around for a while you probably have, you should be aware that every company has sometimes slightly, sometimes drastically different workflows.
So applicants will work with the constraints they know since you are not giving them any documentation and those can be vastly different from whatever you are expecting.
To you an N-gon might be the gravest offense you have ever layed eyes upon, the environment artist might have 5 years of experience working in UE4 and is used to it triangulating on import, while your Engine might crash, erase the subversion history, set the servers on fire and burn down the building as soon as it gets the faintest smell of a pentagonal shape.
Issue NR. 3
Time Constraints
Alright so you are telling me you think a candidate who spent months or years building their Portfolio is going to produce something in 2 weeks that is going to change your view of his abilities?I think that is highly optimistic.
Additionaly you are not only judging a persons artistic skill, but also their ability to carve out time form their lives. Is that really a quality you are looking for in your artists?
Someone who is working his 9-5 to make a living will be at a massive dissadvantage compared to a freelancer who can just take off work for 2 weeks whenever he wants to apply for a job. And if you think someone should take their 10 days of annual leave off to burn themselfes out for your test, for a chance to get a job, I truly have no words for you.
The usual justification for this point is:
We want to see how you perform under pressure.
Thats an easy question to answer I'll do it right here for you:What result do you get if you give an Artist a task but only half the usual time to complete it?
A Bad result.
Corners can be cut, shortcuts can be taken, but is he going to produce his best work? Hell no, but thats what you want to see isn't it?
And this produces another angle of uncertainty, cause ofcourse you don't say where to cut the corners, so Artist A might save time on the lowpoly while Artist B chooses to save time on the Highpoly or the Texture.
So wich one is more forgivalbe? a lacking technical execution or a lacking presentation? Probably neither, but the time has to come from somewhere.
Issue NR. 4
NDAs
So you make people work 14 days for you for free and then they are not even allowed to show what they did.There is a multitude of possible reasons for this that make this simultanously the easiest and hardest issue to fix.
Is it so that people can't "cheat" by looking at other peoples results?
That seems pretty weird, I can look at Michael Angelo's David, that doesn't mean I am able to sculpt it.
Is it because you are sending them a concept from your unanounced Game?
Well maybe use a concept that has already been released to the public.
I don't want people to show it so they can't get an unfair headstart on the Test!
We'll your time constraint is already unfair as lined out in Issue NR.3 so why do you suddenly care about fairness?Additionally you can put everything under NDA except the finished Screenshots, selected and authorized by you. That way future applicants don't know about technical restrictions and chances are they will have to redo most of their work for the test if it wasn't created using your technical specifications.
Issue NR. 5
You don't give feedback
So I have worked on perhaps roughly 20 different Games over the course of my Career, it has NEVER happened to me that the first asset produced for a new project was accepted without any feedback.Thats right, you can give the most detailed specifications in the world, whatever you imagine the Applicant to produce, its basically impossible the result will be exactly like your expectation.
This is where the helpful process of giving feedback comes in handy, you might not get exactly what you wanted in the end, but atleast it will be closer to your expectations then whatever was delivered at first.
+ I imagine your daily routine does involve the application of feedback in some form, so it would be pretty handy to know how well an artist can implement direct feedback before hiring him?
How to Improve your Art test practices:
Okay so maybe you acknowledge now your Art test is not the best to select the most suitable candidates.What can I do to improve it then?
1. Screen your Applicants before sending out a test
This will actually save you time, you should only send out a Test if a Applicant looks promising and his portfolio is only lacking a crucial element you need to ensur the candidate is able to deliver before starting his employment with you.
2. Make sure your instructions are as clear as possible, send censored internal Documentation if possible
If an Applicant is only missing a certain skill you don't think is displayed adequately in her/his Portfolio you should give him a taylored test for that skill, or atleast give him additional instructions to focus on that part of the Test.
Don't just give out the same blanket Test to everyone without any other instructions.
If an Applicant is only missing a certain skill you don't think is displayed adequately in her/his Portfolio you should give him a taylored test for that skill, or atleast give him additional instructions to focus on that part of the Test.
Don't just give out the same blanket Test to everyone without any other instructions.
3. Allow Artists to share the results of the Test
You can specifically point out which shots Artists are allowed to share, so you can ensure they don't show anything revealing about the requirements or your workflow.
You can specifically point out which shots Artists are allowed to share, so you can ensure they don't show anything revealing about the requirements or your workflow.
4. Tailor your NDA to your Test
You should adjust your NDA to allow you to fulfil 3. I suggest putting all provided and produced materials under NDA except the screenshots authorized by you.
You should adjust your NDA to allow you to fulfil 3. I suggest putting all provided and produced materials under NDA except the screenshots authorized by you.
5. Give Feedback
Shockingly I have never seen this done, ideally you should provide 1 round of feedback to promising candidates with atleast a 24 hour deadline to implement it. The feedback doesn't have to be given immideately after completion of the test I'm sure if you provide some feedback at an opportune time within the next month Applicants will still be grateful for it.
Again you don't need to give feedback to everyone, but if you see a promising result but the candidate is just a bit off from your expectation you are doing yourself a disservice by not giving him a chance to adjust his work.
I personally have received feedback after I failed a test, which makes me question even more why it was not given during the test since it could have helped me provide the expected result.
Shockingly I have never seen this done, ideally you should provide 1 round of feedback to promising candidates with atleast a 24 hour deadline to implement it. The feedback doesn't have to be given immideately after completion of the test I'm sure if you provide some feedback at an opportune time within the next month Applicants will still be grateful for it.
Again you don't need to give feedback to everyone, but if you see a promising result but the candidate is just a bit off from your expectation you are doing yourself a disservice by not giving him a chance to adjust his work.
I personally have received feedback after I failed a test, which makes me question even more why it was not given during the test since it could have helped me provide the expected result.
6. Give suggestions for Portfolio improvements instead of a Test
If you applicant doesn't have a good anatomy sculpt in his Portfolio, its highly unlikely he will suddenly be able to produce one in a week.
The same goes for a lot of other skills.
In most of those cases you will help yourself and the appliant more by just giving them some pointers on what you would like to see in their Portfolio and tell them reapply once they have improved their Portfolio (or just pointing them towards common Portfolio Guides like http://wiki.polycount.com/wiki/Portfolio)
If you applicant doesn't have a good anatomy sculpt in his Portfolio, its highly unlikely he will suddenly be able to produce one in a week.
The same goes for a lot of other skills.
In most of those cases you will help yourself and the appliant more by just giving them some pointers on what you would like to see in their Portfolio and tell them reapply once they have improved their Portfolio (or just pointing them towards common Portfolio Guides like http://wiki.polycount.com/wiki/Portfolio)
Alright lastly I will present and counter some common arguments I am anticipating:
All of these suggestions take time, I don't have time
In the end you get what you give, the more time you put into making a better test the better the results you will get and chances are by having a fair testing practice you will also end up selecting more suitable applicants for the Job.
A lot of theese adjustments don't take much time, infact more carefully selecting who you give out tests to will help you save a lot of time reviewing tests and having more specific guidelines will help you find the right applicant faster, again saving you time in total applicants you need to review.
I am giving everyone a fair chance by giving a test to everyone.
Are you even looking at all the tests you get send? Is it faster to download a file, extract it, open it up in the software then just looking at a Website?
If your reply to the first Question is no, then how is that fair if you are not even reviewing everyones work?
The thruth is you are wasting your Applicants time and your time.
If you really are reviewing everyones tests I imagine thats not your favorite activity of the day and you'd rather be doing something else so saving time on that should be in your own interest.
I get 500 Applications a week I need to filter them somehow.
Again, is it faster to download a file, extract it, open it up in the software then just looking at a Website? Also if you get that many applicants why do you even need a Test? If its for some highly specific skill you are unable to judge from peoples Portfolios maybe that makes sense, in any other case it probably does not.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Okay so that was a lot of Text, I understand that for a lot of especially smaller companies its unreasonable to implement all of these adjustments to the process. All I ask is that you consider atleast some of these adjustments and ask yourself if they could enhance your hiring process somehow.
I just want to clarify I am not attacking anyone or any company specifically, this is an industry wide issue. My only interest is to improve the whole process for Applicants and save some of the Reviewers time.
I'm curious to see what you guys think.
Okay so that was a lot of Text, I understand that for a lot of especially smaller companies its unreasonable to implement all of these adjustments to the process. All I ask is that you consider atleast some of these adjustments and ask yourself if they could enhance your hiring process somehow.
I just want to clarify I am not attacking anyone or any company specifically, this is an industry wide issue. My only interest is to improve the whole process for Applicants and save some of the Reviewers time.
I'm curious to see what you guys think.
Replies
Props for writing all this and everything in there seems respectful and legitimate.
It`s nice for once to see critiques and points that needs to be improved from the employer`s side and not the applicant.
Too many times in this industry (and others as well) do we keep on criticizing every step applicants do to get a job and seldom do we ask if the employer`s procedure is ethical or optimized as well.
Unfortunately, employers have so many options, and applicants at the time since they`re in the position of power, they will mos tlikely not care if these tests are fair to each member...
The time thing is a huge issue. On the one hand you want to have time to do polish as much as possible on the other hand putting dozens of hours into a test for free isn't exactly the best use of your time. Those employed only have their spare time, but freelancers are missing out opportunities to put food on the table, so for both its not something that can be done just like that.
I was somewhat lucky with the 4 tests I've got. 2 where regular ~10 days tests (but the company didn't set a hard deadline), one was a 4 day timed test and the last one was roughly about 14 days, but the great thing was it was set up to test how the cooperation with me as a freelancer would work (meaning I could reach out to my lead on a daily basis, I was getting feedback,... the same way it worked afterwards - AND if they liked the work, I would get paid and the asset would make it into the game - by far the best experience as I also got to see if I would want to continue working with them). Oh and I had one small test within the company to check my hard surface skills - they just gave me an elbow pad concept, some examples from the game and told me to make it, which was great, as it only took a couple of hours on a weekend to work on it.
The 4 day timed test was good in the sense of it wasn't taking too much time from the applicants, but even a senior company employee seeing it said that the timeframe was ridiculously shorter than what you regularly would have at one of their studios for the same amount of work (likeness sculpt, sculpt a shirt, pants and boots based on a provided concept, lowpoly of the shirt and textures). Anyone working would have need to sacrifice a weekend + 2 working days to make it.
Except for the freelance gig and the company intern one I was always unemployed during the tests, so I could make them fit. I don't want to know how it is if you ever end up looking for a job and you need to do several tests in a row. Basically no private life and those tests provide no income, they become a huge strain on everything... taking vacations to get a job? If its not my dream job, definitely no chance of me doing it - not out of arrogance but out of necessity for things like family.
I guess by now I don't see the point of big tests. They should be kept to the absolute minimum to remove the last questions about your candidate like the one I've got within the studio I was working. If it takes more then a 2 days (a weekend) then think about re-numeration. Also you need to be flexible on time so your candidates can find the time for it and not be forced to take vacations / (unpaid) days off.
It's also ridiculous when you compare it to what other positions get. My girlfriend got tests as a producer/project manager. They ranged between 1 hour (timed) to a couple of hours. Its crazy how little respect there is for the time people demand from artists. Starting with all the time necessary for portfolios to tests that take several days.
The lack of respect for other's time is what really aggravates me about this whole process. Especially since so many flaws in the art test format come from a myopic, entitled idea that the employer's time is more valuable than the candidate's time. Grr.
I remember seeing a job posting from the Astroneer team a while back that came with a briefing for an optional art test. Basically giving a candidate the opportunity to complete an art test if they felt their current portfolio didn't hit all the right notes stylistically. This seems like a good way to approach it since the candidate could take the initiative if they wanted, but a more experienced artist could just apply and not be dismissed outright if they didn't have the time to do the test.
I do feel that there can be a disconnect between company policy, art test reviewer and the HR coordinating the test simply because of how many AAA company's operate filtering a high volume of applicants for a single position.
There have been suggestions that paying for an artists time to do the test might be a good way to remedy this. They can always write it off as a business expense.
RE: Vague briefings, I think you're missing one of (if not the primary) reasons that art tests are often kept vague:
The employer wants to see if the candidate can think critically, if they can research the subject matter to fill in the blanks, and if they will ask questions to clarify the task (communication is a crucial skill). Essentially, figuring out the test is part of the art test. Personally, I don't like this method (it's possible to design a test to stress these aspects without being vague or mysterious about it), but I understand the rationale behind it.
Giving feedback / portfolio suggestions to every person who applies is a monumental task, if you've ever done portfolio review you would realize that the candidates who get art tests are maybe the top 1%. Art tests are generally not handed out like candy, but of course this can vary per studio. I agree in part here though, if there is a studio that literally hands out art tests to every applicant, that would be excessive and they should stop doing that immediately.
If I was going to push for one thing to change in the world of art tests, it would be: Pay the artist. This cuts out most of the problems with art tests. If you're paying people to do the work, you're going to weed out everyone but the serious candidates. You can put together a more realistic test with NDA sensitive material, and introduce the candidate to the feedback / approval process, to see how the candidate performs in something closer to your day to day production pipeline. If you're paying the artist, the question of whether 2 weeks is enough time or not (usually) goes out the window, they'll get it done, and if they can't with money on the line, that tends to be a red flag. But in any case, it's good to talk to the artist and figure out how much free time they have for the test, as you may be considering someone who has a full time job or other commitments. I tend to get both an hourly quote and an estimated time for completing the work before finalizing the terms of a test. So yeah, treat them like people and pay them for their work. It's usually a good plan, but you have to have leads and people in positions of authority to argue for and agree to it, which can be difficult at larger studios.
When it comes to finding a good fit, whether they can make a decent asset is only half of the equation. When I've been involved in setting up art tests, I've tried to structure them more like an initial contract than a typical art test, to get a better feel for not only their work, but how they are to work with.
But
I honestly can't see many studios paying for a candidate's time - the cost of the admin surrounding it would stomp on that...
Anyway,
I've set a fair number of art tests over the years and I've found them to be extremely helpful in situations where a candidate is clearly skilled but doesn't have work that's representative of the style they'll need to match for the job.
It used to be relatively easy to come up with a test when a decent environment asset could be turned out in 3-4 days but thats not the case anymore and it's totally unreasonable to expect a candidate to put more time than that into a test.
A 4 day test shouldn't affect a graduate - they can use the results in their portfolio and if they're not already putting every spare hour into portfolio work they're not trying hard enough to work for me anyway.
An employed junior is a bit different - the same test over a couple of weekends doesn't seem harsh to me though.
For anyone more senior than that a test won't tell you anything - the portfolio is either good enough or it isn't
I now refuse to ever take another art test. Pretty much for the same reasons as outlined in your post. But also I was handed a test to do, without ever being considered due to relocation / Visa. Another was a test I did for a studio that handed it to me, but I read on Kotaku, the project was shut down. And lastly, I refused the art tests several times, and was still offered the job.
Several years ago I started this thread which was eye opening to me.
https://polycount.com/discussion/128692/how-many-aaa-character-artist-positions-are-there-in-our-industry
We counted less than 500 of us in the world. And even if our estimates were wrong by half (they're probably not) there's still not even 1000 in-studio AAA Character Artists worldwide.
Specifically in my case, there's about 40 of us here in Montreal, and we all know each other, and we see job openings posted here that remain unfilled for years. We know which studios are willing to relocate a Character Artist or obtain a visa, and which ones won't.
The days of recieving hundreds of qualified applicants a day is long gone.
Most Seniors won't bother relocating anymore. We have mortgages, with kids in school, and wives with stable careers.
Maybe this is my naivety, but knowing full well how god damn impossible it is to hire a qualified Char Artist in 2019, I don't think the studios are in any position to make these kinds of demands anymore. But maybe it's also because I feel like I have plenty of high end options that don't require me to take a test.
Our portfolio and experience shows what we can do.
There's a good chance places like Naughty Dog, Blizzard, and Epic get tons of qualified applicants, and my refusal to take an art test might preclude the possibility of me ever working for any of them.
There are more than enough students/beginners who are about the same level, but there aren't not that many who are actually on the level for AAA-character artist positions. JacqueChoi said 'hundred of qualified per day' - which definitely isn't the case. You will have a hard time finding that number of bonafide AAA-portfolios on artstation for a position - not even talking about if they are looking for a job/are interested. You could have 100 applications per day, but if throughout the whole opening process you get 5 proven AAA-artist for 1 position in the end you also have enough applicants, the other hundreds are just oversaturated noise that you are not interested in.
I guess doing art test and the size of it will depend on artist and studio involved and who has more leverage. There are a few studios where I would do whatever it takes to get the job, but at this point in most cases I would also reject any test that requires more than 4-7 days work - or doing a test at all if its just a small studio without a proven track record. All a matter of opportunity costs and those shift depending on your situation.
For 'there is no time' - that's simply not true. I have seniors, leads, art directors around me taking time to give feedback, help me out and things like this. It's not like everybody does his own thing and just crunches and is exploited - your view of the whole industry is tainted by articles and based on an outside perspective.
I've spent a large amount of the last 2-3 months teaching juniors in our studio - in both cases I ended up spending more time on teaching them to reach the required quality than if I had done the tasks myself. I don't mind, but you will do it only for people you see the potential to help out in the long time. The difference is - for intermediates and seniors its enough to take a couple of minutes to look at their stuff, maybe do a in depth review once the tasks are done and drop them a few lines and they will know what to do. With juniors you need to look closer, explain, teach, make sure they understand the big picture... to me doing all of that is fun and I gladly help, but it isn't efficient from a business perspective - not talking about 10-20% less efficiency, talking about losing half a day of a senior to get a day of work from a junior, that still has less quality - now tell me that it doesn't make sense to have an intermediate there that will cost the senior about a few minutes per day to get a similar quality.
Getting back to topic though, there is room for testing applicants, but the more complete the portfolio is, the more experienced the person is, the less should be a need for an art test. And again, the tests should be as short as possible respecting the applicants time. I understand that you don't have time to give in depth feedback for the art test, but there should be a short summary what worked out and what not. You reviewed it, probably made internal notes for your studio, so share it with the candidate.
And actually be honest about what you are testing. If your communication with the applicant is done completely by the HR then don't play games to see what the candidate will ask or not about your briefing. You are basically already telling them that you don't have time to deal with them that's why you keep them limited to the HR - do you really think this invites to ask for clarifications?
I've been working for a big outsourcing studio for the past year, so things are a bit different then at other studios. Being less known than your AAA-clients makes it harder to get staff, so it does pay off to invest in juniors and train them (investing in intermediates and seniors as well), if you see the potential and you expect them to be on industry standard within a year. With outsourcing you also get a big variety of projects and tasks, also simple but tideous tasks that you don't want your top studio artists and art directors to deal with. Outsourcing those to cheaper locations (due to lower costs of living) pays off for both sides and is actually ideal to get juniors used to life in a production environment and quality standards.
By maintaining both clients and staff happy you get an experienced working force, while you can get to the point where you can actually pick and choose your contracts. So yeah, its actually build for long time success and by what I've seen the studio was successful at that. There are always things that aren't perfect or people that leave (me for example), but it was a great place to work, treating their employees absolutely fair (most people stay there for several years).
The upside is, if it works out, all great, payment will be adjusted and the artist gets into the project. the downside is, there will be nothing to show for the artist, if it works out for sure not, if it doesnt work out, then only until the project is published.
Vague Documentation
This has been an absolute nightmare for some art tests I've done. One of the main problems I've run into is art direction, where the technical aspects of the test might have been ok but I've failed to hit whatever vague goal the test reviewer had in their minds. Three real-world examples for me:
- The asset was too clean, it should've looked dirtier
The company had different games on their portfolio with different art styles, maybe should've specified a bit more art direction for the asset? At the very least the game I should've been aiming for?
- The asset was too dirty, it should've looked cleaner
Same as the above.
- You should've used alphas for this part, it was too high poly.
No tri count budget was given and I was expected to treat the asset as a background prop and not a hero asset, but I did the opposite.
These three examples had something in common - no art direction or further specifications were given, how could've I known? I tried to take my best guess but no amount of experience would've helped me here. Luck should not play a part in being a qualifier for hiring someone out. Your feedback phase suggestion makes a lot of sense here, as most of these issues could've been solved in a few hours. Of course, I know how to create alphas and create different types of textures, but once it is turned in you are done.
Common Industry Workflows
I haven't run into too many technical problems during art tests myself but this is a huge point as I imagine a lot of people have and will. I used to work at a big outsourcing company for a few years and got around to working with over 15 different game companies, everything from big AAA ones to indies. One thing I can definitely say is that everyone works completely different! Using a different example than yours, I've seen companies being extremely specific about how they arrange their UVs. Can I mirror UVs? Are tiling textures OK? Can I use trim textures? Should I prioritize UV space on the important parts or does the density need to be completely uniform? Does everything have to be unique? There are dozens of workflows regarding UVs and I almost never see any specifications on them. How is guessing for the correct method a qualifier for hiring someone?
Feedback
There could've been more reasons why I didn't pass those tests that I didn't know about, of course! But without any further feedback than one or two vague sentences, I don't know if that was the case. I know it takes time, but receiving detailed feedback is the very least someone deserves after pouring their hearts out and investing so much time instead of getting a cookie-cutter response.
Failing art tests is extremely disheartening in general - even more, when you feel like your effort went nowhere and you had no idea why or you were just not lucky enough.
Lots of great comments and viewpoints here, keep 'em coming.
I do want to say the point about freelancers just taking 2 weeks to test is a bit unfair. When I was deep in freelancing, a test was 2 weeks I wasn't actively working (or looking for work), and thus not making money. It's just as much of an opportunity cost to me as it is for the person taking PAID VACATION (totally understand the angle of taking vacation to work on a test though!).
I just took my first paid vacation ever and it was great.
This will change in a few weeks, but I literally couldn't take an art test right now. I commute 4 hours a day (roughly) and and exhausted by the end of the day. In the mornings, I keep fit and honestly am not mentally awake first thing in the AM. I tried working on the train part of my commute but my results were mediocre. Without that, working at the same time as doing tests isn't easy and you do lose your social life, but it is something I would probably still do.
There was that game Mass Effect Andromeda that was made in Montreal.
https://www.youtube.com/watch?v=7KWkao73HuU&t=658s
Was the reason for the character work not being up to standard because of the lack of qualified character artists not being available in Montreal at the time or something else? Were any of the 40 character artists you mentioned involved in its development?
Like that one incident liquidated a whole studio. What happened?
I think that's the point @Alex Javor is trying to make, that its important to see the bigger picture and all aspects of project management, maybe the solution would be to take mid level artists and train them to fill the deficit?
Are there fewer character artists at a studio because that's the best available, or is this necessitated by the way games are made today with a lot of the pipeline outsourced?
Some aspects of character art are more challenging than others and there are specifics that are best learned in studio, so being qualified seems to be a very fluid subject. Like qualified in what capacity? And qualified for what job in which studio? Which style of work?
Like if I wanted to make the crazy mecha dino's from horizon zero dawn, sure as a senior I might have several aspects of the pipeline down, but I'm still not likely qualified for the role unless I understood a lot of engineering principles that were vital to the creation of those models.
So an art test in a situation like this does seem logical.
I think senior pricing also plays a role in this.
Horizon was developed over a very long time by a studio that is utterly uncompromising and dedicated to quality - nothing goes out of the door unless it's good enough.
Andromeda was clearly overscoped, under-resourced and piled out of the door before it was ready in order to hit an arbitrary publisher deadline (presumably forced ).
Neither studio is likely to have a significantly better team of artists - the difference is management
And: the ones responsible for managing the art test might not get the required time from their leads for properly responding to everyone. This doesn't make it any better but it shows how much of a complex topic this can be.
It is something they ought to put more resources towards. Like in the medical field, its made busy but when it comes to getting new residents they make it a point to show professionalism since that is what is required of the profession.
It can work here too, if the focus was on the people first and profit later especially in AAA.
Forward thinking leaders will take this stuff seriously. It's not about being mister nice guy, it's about building a business/culture that can last.
If making games is such aprivilege, I'd expect the ideal candidate is not Joe blo college kid whose never done nothing but play games and is eager to please to the point of being spineless, but rather experienced professionals from diverse background who understand how to get work done and solve problems. People who earned their stripes in the real world where failure means a lot more than CEO won't get his bonus this quarter.
It's always the poor people who suffer when the lords play their games, and artist are the poor people. So don't play their games.
People say games is highly risky endeavor, so why then are the big companies treating the business like a child playing monopoly? Need to grow up and act like adults. The risk management model seems to be bass-ackwards.
It's cheaper and more effective to use special forces than blow everything up with the grunts. AAA seems to be following the fast food approach, which I don't think is the right approach for the medium and the market seems to be confirming that.
Personally I wouldn't take any test without either payment or guaranteed 1 on 1 feedback post test. If company can't make that happen, probably not worth working for. What students should realize is, they have the power. It's not that hard to make games. If established companies cannot find a way to treat workers like humans, make your own games. Start your own business. It is easier than trying to get a job in AAA, and you won't be subject to unethical treatment.
My first published game I made with one other person in 10 weeks, zero marketing, its hit trending in its genre on steam and has a small but scalable fan base already. Am I so smart? Not even close, I just don't buy into the BS and do my own thing.
Certainly any motivated AAA vets out there could form small elite teams and be killing it. Just look at rimworld made by a single experienced designer. The essential skill is leadership - not tacky self promotion - which is how you'll make good shit happen for yourself rather than making some rich dickhead richer.
Also a lot of the crap I have made while learning I sell. Dont earn much but buys dinner now and then.
It is very stiff competition, so why take extra risk? Work as a hobby until the money is enough, rather than become starving artist.
In future when asked to do a test I think I will preface with the correspondent that my test has been looked through carefully AND by the same people that will be reviewing the test and if they truly consider me a promising candidate. Otherwise they can forget it because they are asking me to spend a lot of time and effort (for free) on a test of which there is a 90% it will get turned down anyway. It's just becomes a big waste of time... Or maybe I should just ask straight up to be compensated?