Masks (the rgb channels in the video each contain a mask, you could in theory fit 4 masks into one file) and LERPs. Each seperate material needs its own mask, you can chain the lerps together to get the final results.
Question, how can I get the masked blending mode to have smooth edges? it's always rough and hard where the black blends into white, my opacity map is fine and I've never had this problem in udk. translucent blending works just fine but masked is always rough or hard around the edges. this is especially bad for character…
Like ZacD metioned, mask out RGB, do your operations, then do an append node with R and G as the inputs with the output of that going into a second append with B.