License-able Pipeline Tools?

Noth

I don't have a lot of experience here, or exposure but the bit of exposure I do have with the toolset I'm referring to, I'm wondering if it's like this at other studios.

I used to work with a place that had a piece of software they built themselves, you would open it and it would have all of your tasks in it, you would do all your opening Maya scene from it, it would get tasks from Shotgun and you would publish your tasks to the next dept from that tool too, it would update the task state on Shotgun so management would know how things were progressing, etc. 

This tool was helpful because it enforced naming conventions, maintained project folder structure, updated the relevant people as to what was hanging or complete, and notified the people next in line for a task, that if their task was visible in the tool but blocked, it would then be accessible for say the rigger, if the model was just published. This sort of thing only gets more useful the larger the studio gets, because people either laziness or accidents, it removes those largely.

K, so on to my question. Does anyone know of something like this that is flexible enough for game dev? Something that could be taken from project to project, even if they're quite different?

I'm also asking because I would like to learn more about this area in Game Dev, so any insights here is very welcomed. What's your experience like at different studios, things you like / dislike, awesome apps you know about, good things at a studio, bad things at a studio, etc. More the merrier! 

Cheers!

Eric Chadwick

Shotgun has a lot of these features already, but from what I hear it's quite extensible. Usually these kinds of tools are home-grown, and specific to each studio's data layout, DCC integrations, etc.

Eric Chadwick

Did a quick Google for "game project management" and this might help you
https://gamasutra.com/blogs/MeredithHall/20180629/321013/Choosing_A_Project_Management_Tool_For_Game_Development.php

Noth

We use JIRA and Confluence, I don't mean this sort of tool for PM but more asset handling, so everyone isn't saving final_final.fbx, or so that the leads don't have to reinforce naming conventions. I was hoping there might be something flexible and deploy-able someone might know of, I wonder if every studio's needs are so different something like that wouldn't work

oglu

For version control perforce is kinda industry standard. https://www.perforce.com/
It does have Maya and Unreal integration.

Noth

Yeah, we use Perforce too, and I've used a couple others in the past. What do you guys used to handle Source files? all the working files, maya scenes, huge PSDs, marmoset baking scenes, 3DC, Painter, etc. Is that all local then people just push final assets to the project via Perforce? Isn't there a shared drive so you have a backup you can go back to if that person leaves, or someone needs finish the task, etc? How is that portion handled in your exp?

poopipe

Source files go in source control.  To suggest anything else is completely insane - game resources are replaceable, source data is not.
If you're worried about all the different file types, streamline your authoring pipeline and stop letting people use whatever method they fancy to do things. 

As far the Uber system goes...

I'm not going to say it'd be impossible to create a general, full pipeline management tool you could use in any studio but I will say it's pretty unlikely that if you did it'd work out of the box for anyone.

Noth

@poopipe Okay, interesting. So do you guys use Perforce for your Source files, and just keep all that organized in a separate branch from your game project's branch?

oglu

Yes we do but check in the source files only once a day. Saving every five minutes onto perforce would just overload the storage. 

Noth

@oglu Interesting, we have a separate shared/networked drive that clones everything each night, which we can get IT to roll specific things back from if need be. We save all our source iterations there, and work directly from it, connecting to it with a VPN.Sometimes ours has latency issues, which I see as a downside, but we're trying to get our server switched to our city, so it'd be faster then.

Do you see either approach being better/worse? Have you heard of any other approaches?

poopipe

That approach is bad. It's not the worst - but it's bad. 

Put your source files and game resources in the same source control repository and make it compulsory to commit a revision of the source data when a game resource is committed (add checkin scripts to enforce this)

If you don't have a link between revisions you will find yourself in trouble and it will be expensive to dig your way out of it.

Eg. a bug needs fixing in an asset on a release branch and changes have been made to that asset since you branched for release. 
 If you don't know which revision of the source file produced the release asset you are (technical term) fucked  and need to piss away a load of time on working it out and then properly retesting the asset. 

its a common enough scenario  on traditional release schedules and if you're working on a live service type project its standard operating procedure .
the cost in hardware and support for the storage is very unlikely to outweigh the cost in man hours to deal with the fallout.

Noth

We have a few different branches in perforce for different stages of production, we have a main branch everyone commits to for the game project, and then we cherry pick changelists over to a QA branch, then that goes to a Release branch. We also have longer term goal branches for people to work in for big updates. All of which get merged back to the main branch when needed. That's just on the perforce side, then for the artists source files, they're all on a separate shared network drive (in studio server) with some backup/redundancy from IT, but no source control there. Initially I was just asking about how people manage their "working files", part of the pipeline, psd, baking scenes, painting scenes, all the iterations for these, etc. Is this still bad? 

By add source files to the repo, you mean same repo as the game project, but a separate branch just for source items? If this is the case, I am interested in this, because of the ability to check things out, so people don't work over each other, and the ability to work off-line per-se, and commit things when you're ready, so you don't have any latency when saving.

Thanks for sharing info, happy to get some hard criticism back as I'm looking for best/better ways to do things and I've only seen a few studio's approaches for this sort of stuff. 

Cheers

Noth

@poopipe does that mean everyone has all the source content on their local drives?

poopipe

it's generally kept in a separate directory tree from the resources so people can grab only what they need. These days the source assets for a game are easily going to exceed a terabyte by the time it's done so that's an important thing to take into account.

You want it all to be in the same repo/depot so you can reliably associate a source commit with a resource commit, branch things together and so on 

once you have version control for the source data it's just a case of making sure you have buttons to click for checking things out etc. in the apps you use every day and training the team to use it properly- it's surprisingly not obvious to everyone 

gandhics

Even tho you can find "generic" pipeline tool, there is high chance that it miss something you need. If it has all, there is a high chance that the code is bloated. Also Shotgun doesn't automatically save your dcc file for you. It is more of DB that you need to integrate.

It is not that hard to make own tool with some solid folder structure. If all of your team is on the same network. That's the best way to do it.

Noth

@gandhics would you use perforce or some 3rd party source control for that?