Is there a way to determine the accurate texture from any photo precisely?

melviso

So, I have been wondering about this. Say u have something like this;

I just grabbed this from google . Is there a way u can get the original texture colors from a photo or remove the lighting/shadows.
When I do studies, I usually eyeball the texture color and I get very close to it and have a good understanding of lighting and how textures should look when lit.But it would be nice if there is a way to calculate the correct hue, vaule and saturation of a give surface if it is not lit.
Here, the textures are no longer accurate with the blue hue from the sky and yellowish light from the bulbs?
Maybe like a diffuse version with all lighting and shadows removed. I doubt though but decided to ask anyway.

I think this wouldn't be possible as textures are best captured in a overcast sky lighting? It would be awesome if there is some sort of AI that can do this, sort of gauge the accurate color, saturation and value from photo and it reads accurately in different lighting conditions.
Any ideas.

sacboi

https://polycount.com/discussion/154735/photography-tutorial-for-textures

melviso

Thanks for the link. Unfortunately, I am asking if this can be done for a photo already taken without a color checker. Like a photo randomly from the internet. I am kinda doing some research on lighting and color and am wondering if this is possible. How to calculate the true color of a texture from a photo taken and contaminated with bounced lighting, direct lighting, bleed color, e.t.c.
I am guessing it's impossible.
Is there a sort of chart/ vaule range of what color different  materials should have without lighting information.

musashidan

In all honesty, I don't see how that would be possible outside of a controlled, lab-like environment. Not only are there lighting/shadow considerations, but also colour bleed through light bounce. So, in your example, as well as the sky you have the green from the landscape and the different building materials themselves.

The poor man's method is to source refs in idealistic overcast or diffuse conditions and colour pick from that. Or source the actual building materials and go from there. Even so, the timber in your ref - which I'm guessing is Cedar - is naturally a myriad of colours, and even moreso as it weathers and ages it naturally changes shades. Cedar naturally turns grey as it weathers so, in fact, the timber under the eaves is also cedar that hasn't been as exposed to the elements. Off-topic, I know, but still these are the types of things we think about when recreating in CG.

Eric Chadwick

There are hacky ways to do this, see the latest demo video of Allegorthmmic Alchemist for an example of this in practice, removing lighting/shading info from a texture. You end up losing detail.

More info why
https://www.unrealengine.com/en-US/blog/imperfection-for-perfection

melviso

@musashidan And that needs to change. There should be a way this can be decoded. So what about textures u need to take in a subway underground or dark place. A way to decode the exposure of the photo, calculate the light intensity as well as the original color of each surface. I don't know.Maybe there would be something of this sort in future. We are already making significant progress with AI. So maybe that would be possible?

@Eric Chadwick     I  checked it out the latest demo about it. The one they released yesterday. Its cool. But is there much difference between this and designer or painter. Though it seems faster and more hands on. The colors u pick from the images in the demo. They are not accurate. These are colors already contaminated with light, bleed, e.t.c. U might import the textures into the scene and they may be too saturated or muddy due to the difference in lighting between ur scene and lighting in photo sampled. Of course, color correction can be used to tweak this.

There needs to be a way to calibrate lighting in any kind of photo and lighting in a 3d scene, so the right colors are used hence looks correct in any other lighting condition. I also feel overcast lighting isn't good enough for accurate colours as other lighting factors as well as the ones we mentioned still exist somewhat. Maybe I am looking too much into things :- )

As for what I was looking into. It seems I have found some sort of way to get close to what color should be. I am still not very sure if this works in all materials per se.
So I will experiment further. Also another thing to note is the difference btw color picker in photoshop and 3dsmax.

poopipe

melviso said:

There needs to be a way to calibrate lighting in any kind of photo and lighting in a 3d scene, so the right colors are used hence looks correct in any other lighting condition. 

  If you actually think about the problem you'll realise that it's unsolvable and the best you can hope for is an educated guess - Which is what you're doing when you eyeball it. 

You can measure this stuff on site if you take special equipment - you cannot measure it from an LDR photo which has had it's colour profiles buggered around with in photoshop. 

gnoop

For an accurate texture hue, saturation   you could use  just a proper gray card or  x-right color checker passport  https://xritephoto.com/colorchecker-passport-photo  to remove the sky color bleeding etc.  Although I already misplaced mine somewhere because I had hardly used it after all.     Nothing is wrong with just eyeballing it too.  At least our engine does in a way  you have to do some eye based correction anyway. 

Bigger problem is removing lighting /shadows.

There is a nice soft for that  https://lightbrush.org/     

Working not always perfect and needs some time to master, still does wonders.  A single reason I started to shoot RAW files.  

  It couldn't eliminate micro shading from cracks and small details sometimes but generally works pretty acceptable  and without all that mirror balls pain in ...

They started to charge per image recently  but who knows, maybe they would go back to sell it as stand alone again.

melviso

@poopipe Yeah, I guess so. But in future, there might be something. 
@gnoop That looks cool. It seems close to what I am talking about. They stopped selling standalone? I still think the tech is still in its early stages. Needs more research but the software is definitely a start in the right direction.

gnoop

melviso said:

They stopped selling standalone?

Yeah, it was like $1800 first , then suddenly $300  for MAc only. I bought my  first Macmini for that soft.  Then for a few years it looked like the company is dead, nobody even responded for support and finally it resurfaced as a cloud based service.

AlecMoody

I'm not quite sure i understand what the usefulness of a workflow like this would be. Like others have mentioned, if you want an accurate reproduction of a specific surface you are going to want to shoot texture photos with some kind of white balance and exposure reference in the scene. Even better, use some kind of material scanning work flow that accounts for more properties.

In this case since it is a photo from the web, there isn't enough resolution to actually pull textures off of it. Why does it need to be accurate? Intuition tells us that concrete is slightly on the warm side of grey. A little bit of artistic license and you can work your way into an good match.

poopipe

melviso said:

@poopipe Yeah, I guess so. But in future, there might be something. 

No, there won't, my point is that it's impossible to go beyond guesswork because there isn't any concrete information to go on. 
It's like me asking you to tell me what the value of X is in the following equation...
X = Y

It doesn't mean we can't make a good guess with software but don't fool yourself that it'll be physically accurate

bitinn

Don't know any existing tools that can do it well. BUT if you want to know the academia answer, then yes, it's possible and there are many papers on this topic:

eg. https://salman-h-khan.github.io/papers/TPAMI15.pdf

poopipe

@Eric Chadwick
Alchemist has a few extras that aren't in designer yet - better normal to height conversion and delighting are among them.  They'll filter through soon enough though. 

melviso

@gnoop Ok. I have never heard of them until now.

@AlecMoody I am not saying I can't get a good match by eyeballing it. I can do that easily . I am wondering if it is possible which I speculated was not possible in my first post but I think is something we can diccuss and think about. So I made a thread about it. As for height or other map extraction, there are already softwares for that. My gripe is diffuse, correct diffuse from a photo.
Not everyone can afford to travel to different countries and snap photogrammetry pictures for a particular asset/environment if they wanted to. Or say a particular type of expensive furniture, are u gonna go buy one to get the correct textures or go out to a furniture workshop to go snap one. What if this isn't available around your area. U travel? When u could just grab a photo from google and be done with it. 

If u are given a photo to reconstruct a scene say this building has been demolished a couple of years ago, u have to get the diffuse color correctly, there is no way u can get the correct diffuse with photogrammetry since it no longer exist. U have to eyeball it which I am good at but having something like this makes things locked down and more streamlined and the asset can be tested in different lighting conditions and it looks accurate down to the T.
There are softwares like substance designer that enables u to build materials from scratch but that color been sampled from photos isn't accurate. U might have to do a lot of eyebaling to ensure it looks correct in every lighting condition.

Concrete has different kinds of hues, saturation and vaules, from age, weather, moss, rust or painted color so, starting with warm grey wouldn't work in every situation.

@poopipe U have a point, I agree but maybe 2050, this is possible!! :- )

@bitinn Thanks for the link, mate. Nice to know some research is being done in this area.

At the end of the day, it is not possible, Eyeballing is your best bet but it is always good to raise a point about something u find interesting while working in cg. U never know, the future might decide to surprise us or maybe not.

gnoop

bitinn said:

Don't know any existing tools that can do it well. BUT if you want to know the academia answer, then yes, it's possible and there are many papers on this topic:

eg. https://salman-h-khan.github.io/papers/TPAMI15.pdf

Lightbrush does it pretty well actually , but not for every subject and you need to get accustomed to it  since lots depend on your initial  manual input.     Still I  wonder why it's not in something like Photoshop yet   and nobody seems interested.      Looks like company is half dead and struggles to find customers while imo could provide a revolution in image editing and texture making.     It should be a part of any RAW camera processing  by now together with rough depth capture through IK sensor or something