Wobbly lines in my VR 180

Ruz

So I have more or less solved how to make decent 180 VR , but this infuriating problem has cropped up now
https://www.youtube.com/watch?v=ejqhof5034k
The lines are all wobbly, almost like you tube did not quite manage to read the format correctly.
works fine for 360 stereoscopic
look at the lines on the walls and the roof, wobbly as hell
EDit, it must be premiere doing this as it looks the same on my desktop, weird

Ruz

mm, still no joy, not sure what the problem is, does the same thing with VR180 creator

Ruz

may have solved it by rendering each eye at 1920x2160 ie non square, but still 'almost' square
but when you place them side by side they come out to 3840 x2160 which is what you tube expects. In my first experiment it seemed to fix the issue, but have to test some more with higher res images

nahh still broke

Eric Chadwick

Can you screenshot a closeup of the problem? Not seeing it here on my phone.

Ruz

EDIT It seems that it does has the same problem on DEO VR so it's NOT you tube specific. It seems that
This only happens when I inject the metadata to tell that the video is VR either by using Premiere or Google VR 180 Creator
The probem is that if I don't use the metadata, you tube will not display it as VR but DEO VR will detect it automatically
it would be good to have it on both platforms as DEO VR has great VR controls
This problem does not affect 360 stereoscopic or 180 mono

Ruz

I can view as a still image locally and it looks fine, no wobblyness, it's only when I convert it to a video and add metadata and upload it that the problem appears( mp4 looks fine locally)
here is a still image before any processing. DEO VR can read this properly as is

Ruz

I noticed the same problem on other VR videos on You Tube Here is one of Venice. I wonder if that's why google removed the app that injects metadata
and maybe this is just something I will have to live with?
You can see the same wobbliness when you are in the VR headset, so it's not jsut like its displaying it weird in 2d

Ruz

I am wondering if it's the cameras x and y should be locked at 90 and 0 when rendering? will try that next, but I am not hopeful

Eric Chadwick

I bet it’s a fault inherent in YouTube’s projection transform, so there’s not much you can do about it.

I wonder if they’re wrapping your renders onto a sphere geometry, but it’s fairly low resolution.

Ruz

I see what you mean, because the wobbles are regular and appear faceted in some way, almost like a low poly sphere
I have a hack to fix it ie export the video as a 360 , but crop it to the centre part of your screen, so it looks exactly like a 180 VR
It's just that i want to draw to viewers eye to the front, the side bits are not really needed, unless there were amazing things going on.

This is the hack, which although is not 180 sbs, is still looking like a 180 VR. Not sure if the dead black space takes up much extra memory
still annoying though

https://youtube.com/shorts/ex1mKDoz9uc
It might just be my eyes, but it seems there are still slight wobbles, but much less distinctive, maybe they are just repeated more, but the edges of some signs do have  a slight wobbly feel

pior

This is really interesting. I wonder if uploading something made of very thin black and white parallel lines would reveal what's actually happening ? Perhaps the resulting moiré would reveal the projection sphere, if any.

Ruz

Indeed, i might give that go this afternoon. I did email google, but not  sure if they will get back and fix the issue

It would be really bad if you were making a tron like video if lots of grids

Ruz

hmm, this test seems to show that it's the poles that are the problem, looking staight ahead, the lines are 'relatively' straight. but as you look
down to the floor the lines get more and more messed up, same with the ceiling

https://youtu.be/baLqsWV7t0U

something to do with pole merge settings which i don't quite understand

pior

If you screengrab the footage you can actually see curvature of the sphere quite clearly within the black areas : 


So I think two more tests could give it away : uploading a fully flat mid grey render (which would reveal any shading that the 3D rendering of the sphere adds to the footage) ; and, a render made of very thin scanlines (which would likely reveal the triangles through faint but noticeable shifts of direction compared to the horizontal lines in the source).

Ruz

ok, cool will look at that some more, but I have ran out of you tube uploads for today
Kind of weird though that the distortion on the 2 squares on your screen grab near the bottom are more extreme than the one in the middle( bottom)
I am not technical enough to figure out what is going on here tbh
It must be the way they render it online, becuse it doesn't matter what I do they look fine as mp4's offline on my local drive and as still images.
I have not seen even 1 video on you tube that is VR 180 Stereogrpahic that does not look wonkey, just most people don't really notice the issue or some of the videos are such poor qulality that its hard to see

Ruz

might just have to live with it for now or do my 180 hack

pior

As a matter of fact, the "sphere artefacts" I noticed earlier seem to be encoded in the video and not caused by the realtime rendering itself - as lower res playback makes them more obvious. So there are probably two factors of degradation hapenning at once here : artefacts being encoded in the video, and wobbles caused by the reprojection onto a 3D sphere during playback.
So perhaps the sphere projection used for encoding is ever so slightly different than the one used for projection/realtime rendering (different geo density and/or different triangulation).

Ruz

It's odd that google withdrew their app VR180 creator a while back, maybe they just broke it
The worst aspect for me is the way it creates waves in the image, really not acceptable for high quality VR playback
i did see a video done with Davinci resolve which seemed not to be displaying the warps. but its not displayed in VR in the video, so it probably is warped. will check that out
https://www.youtube.com/watch?v=ZwojozYr68M&t=842s

when you mention triangulation, maybe the triangle edges are pointing in different directions? Like when you do a low poly model and start turning edges so that the slight texture distortion you get  is mitigated by turning edges? Not sure if that make sense.
need to hack in to google server and replace the low poly  sphere they use

yeah same as the other ones

pior

I think the triangulation could be a factor, but perhaps not as much as a mismatch between the density of the sphere used for encoding and that of the sphere used for 3d display.
These artefacts should be rather straightforward to reproduce (for the sake of clearly illustrating the issue), by projecting a scene to a sphere of a given density, and the result onto a realtime 3D sphere of higher or lower density ...

Ruz

you would think google would know what they were doing with all that expertise and money on hand and i am surpised that  no one else has even noticed ths issue, bar one guy who I can't find the link to now
 They changed the VR format a while back in to a kind of weird cube format that no one else is using. Some reckon it's to stop
people watching them offline

https://github.com/ytdl-org/youtube-dl/issues/15267

Ruz

I have reproduced it as you mentioned, mapped a 360 image to a low poly sphere( not sure if this is what you had in mind)
The effect is very similar, until you subdivide it a few times , then it goes away. think you solved it 
Doubt they will fix it any time soon though. I am thinking if they used a smoother sphere then the resolution would be a bit sharper
as they pixels would not stretch to fit the uv's as much, causing a 'mushy' effect

Ruz

more info
https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md
https://groups.google.com/g/spatial-media-discuss

bit programmery for me

poopipe

fwiw this sort of thing is inherent to any sort of cartographic to spherical transform that doesn't have infinite (or at least subpixel)  resolution
As you surmise from subdividing the sphere, more sample points = less artefacting - but only because it made the artefacts small enough that you can't see them. 


That youtube stuff looks rough as hell. 
I'm not convinced that sphere is a simple sphere and suspect it might be one of those helpful irregular remeshed spheres programmers are fond of. it looks quite high density to me 
if you did the same thing but instead of the b&w grid, used different coloured thin stripes it might be easier to pick out where the vertices lie (at the pointy bits) 

Ruz

I did find a few vids that were pretty ok in terms of distortion on the DEO VR site, but probably they came from
cameras that used fisheye and they were converted to equiangular 180 VR with the software that came with the camera.
real pain in the butt tbh
I can render from blender with fisheye but then its bloody hard to find good software to convert them and I am still
not sure that it will get rid of the distortion (sigh)
can of worms etc

Ruz

interesting article
http://paulbourke.net/dome/fish2/

Ruz

So i tried different methods like rendering dual fisheye then converting to equirectangular, but they all pretty much look the same once uploaded.

It's either Youtubes fault( probably) or the metadata that has to be injected in to the file before upload
You can either do that in premiere by checking 'video is VR' or with 'VR180 creator' which does pretty much the same thing

Will just have to make do for now, wobbly shit FTW

Ruz

so you can quite clearly see the sphere they project on to in this image

poopipe

oh right - that's a boring sphere and I was wrong then

Ruz

thya could have at least smoothed it. I am curious why they need to do it this way anyway in that surely they can just take the uploaded image and show as is, why reproject it?
I have seen other stuff on DEO VR where this does not happen at all. Would be good to know what is in the metadata that tells you tube that it is VR

pior

"surely they can just take the uploaded image and show as is, why reproject it?"

Now you lost me ... Sure enough, the source 2D image for this kind of content is similar to what an unwrapped sphere looks like. They're not *re*projecting it, they are simply applying it to a 3D sphere to display it and allow the user to freely look around ...

For the sake of clarity, could you show what a frame of the raw, 2d footage looks like ?

Ruz

ah ok, maybe just semantics, but that what i meant anyway  - but surely they can display a higher res sphere than the one they have ? I mean with modern video cards etc
I don't really know how VR players work under the hood, so its hard to speak authoritatively on the subject. Maybe there are 2 spheres, 1 for each eye?
My eyes are certainly not faceted
It might be fun to render out a sphere with a certain kind of shading to see more clearly when displayed as VR
I output as PNG , but this was too big to upload so saved it as a jpeg.

all of the images look fine offline as I mentioned, ie lines are totally straight.
You can view this image in the DEO VR player and it looks like I would expect, but as soon as you upload it anywhere, whether it be YOUTUBE  or DEOVR it breaks
is it possible that the METADATA injector is broken? so that info that tells the VRplayer how to display the image is wrong or we just right in assuming the low poly sphere senario is responsible

pior

Right, so these are exactly like one would expect - good old images that are meant to be applied to a UVed sphere. So from there, it makes sense that display of these for VR would use precisely that.
I don't think it's the upload itself that breaks ; it would be more simple to assume that the difference between players is simply caused by a different sphere used for the final display.
All this could be clarified by forcing these players to display a raw wireframe pass. It's all quite frustrating because indeed, using a sphere of 2x or 4x the density would have close to no impact on performance ...

Ruz

I think google are a law unto themselves on this, I doubt they will get back to me on this issue.
They already annoyed people by using some kind of unique format for displaying 180 VR, some sort of weird cube mapping, which they say makes the videos look sharper.
So yeah for now just have to live with it, though i enjoyed looking more in to the subject, I learned quite bit.
Its  a bit frustrating really, just want to show the best images possible, but like in games you always seems to have to make compromises re quality etc

Ruz

So I think there is a way around this, not for you tube, but with DEO VR you can view your video as side by side and 'flat'

What this does is make it like looking though a picture frame/letter box instead of being 'in' the scene and this seems to get rid of the wobbly effect

The good thing is that this particular view seems to enhance the VR depth effect