Photogrammetry Textures
greentooth
LATEST UPDATE :
Get my tutorial on Gumroad
My rock textures have been released on UE4 Marketplace
Marmoset viewer for each texture available on my Artstation
http://www.youtube.com/watch?v=reactn7UgRY
________________________________________________________________________________________________________________________________
Hello Polycounters !
It's been almost a year since I started playing with photogrammetry.
My endgoal was to be able to get fully tiling delighted PBR textures out of photogrammetry.
I experimented with quiet a few techniques over the months until I got a good workflow from shooting to low poly.
Now I'll just start building a library of textures out photogrammetry.
Here is the first batch, all the textures are 2048².
Feel free to give any feedback and/or ask questions about the workflow
On a side note, what would be the best way to display these textures ? I love the rounded cube, but it gives a rather noticable seam with tessellation on ...
Get my tutorial on Gumroad
My rock textures have been released on UE4 Marketplace
Marmoset viewer for each texture available on my Artstation
http://www.youtube.com/watch?v=reactn7UgRY________________________________________________________________________________________________________________________________
Hello Polycounters !
It's been almost a year since I started playing with photogrammetry.
My endgoal was to be able to get fully tiling delighted PBR textures out of photogrammetry.
I experimented with quiet a few techniques over the months until I got a good workflow from shooting to low poly.
Now I'll just start building a library of textures out photogrammetry.
Here is the first batch, all the textures are 2048².
Feel free to give any feedback and/or ask questions about the workflow
On a side note, what would be the best way to display these textures ? I love the rounded cube, but it gives a rather noticable seam with tessellation on ...
Replies
Also, what's your workflow for getting rid of lighting information? I saw the video on how Epic did it for the kite demo, but I don't think the software they used is publicly available.
- Usually when I'm done processing my texture in Agisoft, I don't have an area with a perfect square that I can just bake. To get around this I just draw a low-poly surface (usually in 3D Coat) This surface doesn't have to be a square at alll, it's purpose is to get rid of the very low-freq height information and to get all the important bits of texture you want to keep.
- I bake this in a very big (16k) texture (height and diffuse).
- Then in Photoshop I tile the diffuse and height out of the bake at the same time, applying the same transform and masks on each. It can be a bit difficult but so far it's the best way I found to make it tile. What I'd advice is to select the biggest square area you can out of the bake and make it tile with bits outside of that square. You might need to adjust a bit the heightmap brightness where you stamp. I usually focus on making the diffuse right while applying the same transforms/masks on the height, then when everything tiles I slightly adjust the height brightness/mask.
- Finally you just displace your tiling height map in Zbrush, sculpt if you want and bake the normal, AO, etc ...
The whole process shouldn't take more than 30 minutes if your texture has no regular patterns, tiling bricks is much more time consuming with that method...What I really like about this is that you have all the bits of texture you wanted to keep even if they were outside of what you expected to be your texture.
I realize it's not very easy to explain without pictures, I'll try do do an illustrated explanation during the week
As for the delighting, I tried a technique similar as what epic did : bake your HDR sky onto the High-poly, bake that down as a lightmap and invert it on Photoshop. It works even though it's definitely not as good as what Epic did. The issue is that it's quiet time consuming (both shooting and baking). I'd recommend it on object with complex geometry, with a lot of seams on which you cannot fix it by hand in Photoshop. On a flat plane like we don't have big variation in the lighting so it's way quicker to fix it by hand in Photoshop, especially since that means you don't have to tile the lightmap along the diffuse/height.
All of my above texture have been de-lighted by hand.
The most important tip I would give to avoid de-lighting for hours is to shoot under a cloudy sky, or shoot something that is in the shadows, to avoid any sharp direct lighting.
Hope it helps !
I want to share a small tip about editing several maps the same way, that saves me a lot of time. When I start editing the first map I start recording an action in photoshop. When I'm done with editing I stop the action. Then I simply select the next layer, run recorded action and watch some magic
Karynatrychyk, I've tried your method too, but the main problem I run into is that the clone and heal tools can't be recorded.
The only downside of this is that you have to assume everything will be fine while working on one of the two textures, and then apply everything on the second one.
Even though I focus more on the diffuse during the tiling process, I like to toggle the height on and off, just to see if it will match with minor adjustments or if I need to use another bit of texture.
It might not be an issue though, I'll have a try
John Baxter said: Thanks for the explanation, Sebvhe. I'll try it when I get the chance!
Karynatrychyk, I've tried your method too, but the main problem I run into is that the clone and heal tools can't be recorded. I tend not to use the clone and heal tool during the tiling process, because you simply cannot match the diffuse and height with it. You can fix everything just with layer masks.
maybe my Tutorial can help on how to create Tileable Textures from multiple non-tileable Maps at once:
I used the Technique in the Tutorial for the Treebark Material on my Artstation.
Question about your Textures:
Are the Highpoly´s Noisy when you zoom in?
Greetings,
Gazu
The only difference is that the input texture I tile is not a square but has the shape of my retopo plane. But the process is still pretty much the same the same
Yes my highpoly is usually pretty noisy. But once baked down to a 2048² most of it goes away. Creating a hi-freq normal out of the diffuse also help quiet a lot. In the end, if you zoom in on my textures you won't notice any noise from the scan.
I did 3 more this week end. Now that I processed all the scans I made, I'll need to wait for my camera to get repaired and I'll go back shooting some more !
Cheers !
If you can do 3 of those in a weekend, then I would love to know a little bit more of your workflow as well!! I find very relaxing going out and taking some photos, it is really nice to forget about the computer for a while!!
Very good results ;D I love them!
I'm actually not doing them in one week end, well it depends on what you call "doing". I usually process the scans during the week, it takes about 6-10 hours per scan in Agisoft. I'd say it only takes 3-5 hours of "active" time for each texture.
That's because there's actually nothing fancy going on in my workflow
Tricky question, it really depends. For textures (meaning flat area) it's usually better to "translate" your camera while shooting top down. By doing so Agisoft will use the parallax between each shots to create the depth.
A nice example is the gravel texture (5th in the 1st post). I wasn't planning on using photogrammetry for that one, I just shot a few (12 actually) top down pictures hoping to photomerge them in photoshop. Since it didn't worked I processed them in Agisoft and got one of my best results so far. With only 12 pictures instead of 50-100 !
But in the end it's better to have too much pictures than not enough.
Hope it helps !
I might consider doing a tutorial while making my next texture, but it will take some time since my camera is broken.
Then you can create a LowPoly Cylinder which fits the HighPoly Agi Cylinder. Make your UVs so that the 0-1 Space is completely filled.
When you now bake you will get Non-Tileable maps
I'll show you with my next texture
You just bake it in xnormal, using "bake base texture", just like you bake normals from highpoly to low poly. Your lowpoly needs to roughly match the highpoly either way.
You can get it on Gumroad for free here
I also included the texture I'm making in this tutorial ! Here is a screenshot of it
Furthermore I added a marmoset viewer for each texture I previously made on my Artstation
If you have any question regarding my tutorial please let me know !
My main criteria/question is.. Is there any reason you don't use the other > high pass filter for fixing contrast and tiling issues? It tends to work wonders on ordinary seamless textures after the initial tiling.
I will definitelly have a look later today since I'm very interested on photogrametry
okornyal said: Cool tutorial SEBVHE, although i think the process is not complete until a painting application gets a plugin that allows you to use the clone tool on multiple maps at the same time.
My main criteria/question is.. Is there any reason you don't use the other > high pass filter for fixing contrast and tiling issues? It tends to work wonders on ordinary seamless textures after the initial tiling.
Yes I completely agree about the clone tool, would be awesome !
And yes you could very well use a high pass filter to fix the contrast but, the reason I don't use it is because I really dislike the way it affects the texture's colors when I only want to fix the luminance.
I have a doubt about the photogrammetry software. Does it arrange the position of the photos automatically or it is the user who has to input them? I'm guessing and hoping that it is the software who does it. If so, how the hell does he manage to do that? Do I have to activate some kind of metadata in the camera?
I'm definitelly gona give it a try this weekend, if the weather is "nice"
Thank you very muuch
n4uj
You don't need anything other than an object to photograph with good trackable features and practically any camera. The software will do the rest. From there it's all up to the artist learning techniques to yield better results.
Sorry for hijacking!
Furthermore everytime you take a picture EXIF metadata is stored. It contains all the needed information for Agisoft to calibrate the images (cancel the lens distortion for instance). Basically you don't need to worry about that, but if you edit your images before sending them to Agisoft you need to make sure the format you use support it (e.g. not PNG)
I have been playing with photogrammetry but I got awful results xD.
When you are taking pictures, how do you go with the exposure change? If you are using a dslr in manual mode, and lets say you keep the same Iso and the same focal lenght, depending on the area you are shooting at and also depending on the weather, the exposure might change. Then what do you do? change the aperture? Because while I was taking pictures of an indoors banana I got this problem of getting different expositions depending on where I was positioned.
Also, I'm guessing that it is not recommended to change the focal lenght right?
Thank you very much
n4uj
Generally you shouldn't be taking photos in such a long span of time that the sun/clouds would change, so your exposure should stay pretty much the same.
Any tiny mistakes you get you could still correct in photoshop. In general for indoors photogrammetry you want to build a lightbox and a turntable to get perfect results.
You said that the weather is awful, that it's wet and cast shadows.. that's kind of a contradiction, since awful weather in general doesn't have cast shadows due to the clouds that produce rain would diffuse direct shadows. So unless you went out and took photos directly after a rain as the clouds dissipated and the sun came out.. Well, you just made a wrong decision to go out at that time!
I have an old dslr, and the light meter display is a little bit crazy to be honest xD. So lets say I get a range of +-1 exposure in all my shoots, then I should tweak them in photoshop before chucking them into Agisoft? I was worried about tweaking stuff manually, since it is not very recommended... and tedious.
Is there any way to make this automatically in photoshop? a plug in, action or script that takes the exposure of a group of pictures directly to a similar range?
What I meant is that it was raining all night and during the morning it was all wet and a bit cloudy, but yet there were also cast shadows, so I couldn't take any pictures... and now it is ultra shinny, so I will probably wait until twilight ^^
What a crazy weather!! Last night was almost snowing!!
Yeah, I went to Oxford today to do some photogrammetry, the weather was very very capricious, but it's also a good challenge, I'll see how I can save my shots
Also I want to say that unless you have 2 workstations, it is a pitty to make photogrammetry on weekends because it takes all the resources of your computers and rendering it useless... for quite a lot of hours.... it is the main problem I have now with my short experience on photogrammetry. However I will set it to do the stuff while I'm in the studio during working days.
I'm waiting now for a dense mesh to finishhhh
It's been a long time since I last posted but I didn't stop photogrammetry at all !
I've been to Oxford, Bradgate Park and Sicily to do these new textures.
If you want to start or practise photogrammetry I would definitely recommand going to Bradgate Park, Leicester if you're in the UK, it's by far the best place for photogrammetry I've been to. You get dramatic rocks, forests, brick walls, cobblestone, gravel, dirt etc... at walkable distance.
As always, Marmoset Viewer for each texture available at my Artstation
More coming soon !
Since it's quick becoming summertime where direct sunlight is starting to be the default lighting condition... Has anyone tried taking photos with a flash?
Say you can go to 1/2000 exposure in direct sunlight to get balanced photos, but of course there will be heavy shadows on one side of your objects, making it impossible do scan. What if you used a flash to light up that size? There may still be a difference between the 2 sides, but you could at least try and photograph it without changing your exposure settings.
Alternatively, what if you had one of those big light blockers you could carry with you, in order to block the direct light? Just a sheet on a tripod basically. It may be enough for all less than human sized things you'd want to scan.
I'm saying like, 1/200 - 1/100 (shaded side) VS 1/4000-1/2000 (lit side). That means that the lit side will be 10x as bright if it's visible, compared to the shaded side, i think trying to balance it out with a flash could be at least a bit better. The transition between the two sides would probably be still very bad.
I think a portable shade could do wonders, that's why i'm wondering if anyone has tried it.
It would be amazing to be able to photograph stuff when there is direct sunlight, if you are able to block that direct sunlight somehow, since all the light bounces will give you enough light to get good photos.
My biggest concern with it is that it might really mess with all the specular reflections. Everything as specular reflection, photogrammetry only works when it's quiet faint (sand, ground, some not too shiny rocks ...)
The portable shade is the perfect solution. Even though, direct sunlight is actually not too much of an issue, what I usually capture is what's in the shadows.
Try to avoid noon and rather go in the morning/afternoon so the sun is not vertical. That way, you will always get a side, of a wall for instance, in the shadow, shadow that will be cast over the ground/rock you might want to scan. It sometimes gets frustrating but it's never impossible. I went to Sicily two months ago, and the sun never prevented me from taking photogrammetry, I just had to be a bit clever at times.
The portable shade however would allow you to move the shadow with you !
I'll take this opportunity to post a few new texture I've been making. As always, Marmoset viewer is available on my Artstation.
Are you using the record action (clone stamp) method for tiling your multi-image maps?
Also, what maps do you export? Diffuse+normal+height?
I'm indeed using the record action, however not with the clone stamp.
I export both the diffuse and height. Then I tile both of them at the same time (with record action) and displace the height again to bake all other maps.
The reason I'm not using the clone stamp is because it may create differences between the height and diffuse even though you use record action. It's the way it blends the result that causes this. To be honest it's probably not an issue, but I get perfect tiling by just using other bits of texture and transform/mask them anyway. So I'm mostly recording transforms since they can get quiet complex (multiple wraps for instance)
Everything is explained in details in my tutorial
but im really not sure about this here:
I can assure you this is correct. This image comes from Agisoft Photoscan documentation pdf.
And from what I've tried it really works. It's because Agisoft gets most of the depth from the parallax between shots.
Generally you should make a distinction between your type of surface; whether or not you are photographing something flat or something you want to walk around. Either way, standing still and rotating your camera is not something you want to do, and for a flat fall you don't want to rotate your camera even when you move. DO make sure that there is overlap between the photos whenever you move; a maximum of 50% and about a minimum of 25% of the images should overlap around their edges, because photogrammetry uses that overlap to track the pixels in the images.
As for amount of photos, i generally try to make an orbit around my object at 3 levels, if it's roughly human sized. I take a ~20 photo circle around normal height, a ~15 photo circle around ground height, and another ~15 photo circle around head height or higher, if possible. I certainly have some ways to go though, here is what i managed to make so far: LINK
All my models are optimized for games though, so they're not quite hyperdetailed.
I tried to make a texture seamless with no luck, I'm trying to use recorded actions because they just seem like exactly what I'd need.
I got your tutorial when it was released, but it only mentioned recorded actions, and i can't seem to get them to work right.
The clone tool only seems to want to sample one layer, i can't apply it to another layer without a hack involving renaming the layer.
Additionally the big problem is that it only seems to be recording the sample point of the clone stamp, but not the actual strokes.
Any ideas?