Cryptomatte instead of matID in Substance Designer?

gnoop

Have anyone ever managed  to do cryptomatte  AOV  reader in SD?

Is it at all possible ?      If I understand it right  Cryptomatter writes/stores  somehow  multiple ID color values  and  its weight  into same  pixel.   

I never managed to learn how further   extraction goes exactly.   My guess it should be using some range separation or something  to extract masks ( i have no idea really)

Does anyone managed  to do it in SD?     is it possible it all  in its pixel processor?      I once tried a typical idea  of combining depth and color value  into same 32 bit bitmap pixel  and then splitting  depth out  and it haven't  gone very well showing me some weird artifacts ? 

poopipe

what does the cryptomatte actually represent ? is it just a way of storing arbitrary custom data in an image? 

there's no reason you can't combine depth and color into the same image - you've got 4 channels, they're not compressed provided you're using a sensible format (i.e. exr) so you shouldn't see any channel crosstalk or corruption of the data

if multiple channels are packed into a single channel it will be a problem - designer doesn't have tools for bit manipulation so encoding/decoding this sort of thing is kind of not possible (i think it is possible, I just haven't managed to do it)

if you want to post some sample data up here and explain what data is stored where i can have a poke at it - its something I've wanted an excuse to look into for a while 

pior

I am actually quite interesting in this too, on the Photoshop side of things. Where can one download some sample renders/passes in that format ?

gnoop

pior said:

I am actually quite interesting in this too, on the Photoshop side of things. Where can one download some sample renders/passes in that format ?

You mean cryptomatte ?   It's a typical AOV , render pass or render element whatever 3d soft calls it   any modern render  is  doing now instead of ancient mat or object ID.      I attached one I exported from Octane render  that Photoshop EXr-IO plugin opens as several  black&white layers representing material masks   ( instead of weird single rgba color layer without this plugin).   Something I'd like to do in SD directly.

poopipe said:

what does the cryptomatte actually represent ? is it just a way of storing arbitrary custom data in an image?

From  this :      https://raw.githubusercontent.com/Psyop/Cryptomatte/master/specification/IDmattes_poster.pdf

it looks like it store multiple ID colors  and weight  of each such color for a pixel.   But how they store it is still a mystery for me.    They say something about "rank" . I am totally missing that point.    Either how exr stores all those data  and how they are extracted?

Since exr typically shows a cryptomatte as one rgb image/layer withing  multi layered exr my guess it's all stored in 4 channels .  But  I could be wrong  and  it's  actually gazillion separate channels for each and every ID index/color.    I can't find an answer so far.   Either how so called "deep" exr  stores the whole 3d space  in single 2d image  pixel values.   Hope somebody can enlighten me.   like 3d texture ?

I did  managed  to combine color and height  into same 32  bit pixel value and then split  it back  after usual depth "max" combine in old "tile sampler"   by storing the height in  1,2,3, up to 30000  kind of integer values  , adding  typical 0-1  values of one of UV channels   then separating 0-1 UV channel back  after tile sampler . 

Using the height as a "carrier wave"  with around 30000 amplitude   and  subtracting  same 30000 value  in the end to get back the  0-1 channel. 

  

    Still  I  can't imagine how cryptomatte ( if it's doing something like this instead of gazilion separate channels we somehow don't see)   could possibly store more data in one rgba pixel.   Or it  doesn't really do it?

poopipe

Decoding that is decidedly non-trivial.  

It may be possible to generate a substance graph to do it but I suspect youd have to make a new one for every different image. 

If you need masks out of it I'd suggest generating them in your renderer 

gnoop

Yeah. Thanks poopipe     Exr-IO is a great plugin after all .   

Mink

EXR has pretty wide capabilities for image layers and depth, but why use substance for rendering cryptomattes? Even blender has crypto matte compatibility now, and importing whatever you have can quickly be imported for cryptomatte render, which does not benefit from texturing in any way on its own.

gnoop

Mink said:

EXR has pretty wide capabilities for image layers and depth, but why use substance for rendering cryptomattes? Even blender has crypto matte compatibility now, and importing whatever you have can quickly be imported for cryptomatte render, which does not benefit from texturing in any way on its own.

I use Octane standalone  and Blender cryptomatte node can't read standalone cryptomatte, only what is rendered from Blender.    Photoshop can .       Since the purpose was compositing  with  SD procedural stuff  I hoped  I could do it in SD itself.    

ALiepins

It's been a year, anyone got any new news on Cryptomatte for substance?

I very tired of having to re-setup my ID map selections when there is a model update and I need to re-bake. Cryptomatte stores object data and will not forgot what mask layer its assigned to unless the object names change. This is immensely helpful in working in tandem with modeling during production.

I'm not a programmer so I'm kinda of relying on the community for this one :D