those lines do curve, far more than a digital render from a game would. that photo shows a low field of view, which doesnt show the effect as much as a high field of view, but the effect is still there, those floor lines would not stay horizontal for very long if the image was extended. the curve would become even more noticable.
the effect exists regardless what lens you use, it happens any time an image is taken from a point. the only exception would be orthographic images, like from a scanner, which doesnt take the image from a point, but from many points on a plane.
Wait, I was under the impression that games were trying to achieve realistic eye effects and not camera based, and that any effect (such as double lens flare based upon distance) was the artists being lazy and stylish without proper understand of how the human eye looks.
Unless I'm talking outside of my arse, my eyes don't bend object to the extent that picture does.
Not many painters or graphic artists pull out a french curve when trying to draw straight lines in perspective. We pull out the straight edge ruler. It's not just because it's easier, it's because perspective distortion gives a specific effect that makes you feel uneasy or thrown off. It's used more in images of horror or terror than anywhere else for a reason.
If you want to add distance-based curvature then you can do it using a vertex shader or a post-process, and it'll give you exactly the effect you want. The vertex shader version will need you to have evenly balanced verts or else it'll fail to bend long objects, and the pixel shader will probably give a blurred effect. But if you're trying to make a game whose visual style relies on a curved look then it'd totally work.
The standard projection method may not be accurate but its good enough. You can get large field of views by just transforming the vertices in the vertex program (and making sure you have enough geometry for it to hold up, probably with tessellation). The only reason to do this is non user facing data like 2 dome point light shadow maps, or if you are targeting a a nonstandard display device, which no games do currently. This may become more common as this 3d goggle gimmick catches on, but that will be an edge case for the foreseeable future. When viewing a regular screen you have a standard FoV size. If you want some fisheye then you can add that through vertex distortion or post process, but its not going to make the image look more realistic.
but thats the problem, people seem to think that perfectly straight lines are a lack of distortion, but its actually a distortion in the opposite way. a distortionless projection would have slightly curved lines. the farther somthing is away or the lower the FOV, the straighter the lines would become, but they would never become totally straight unless they cross the horizon (or some other major circle an equal distance from your point of view)
im not suggesting games all become skateboarder handy cams, but fish eye is a natural effect that is always there. is not an optional effect you add, it is the natural reality of viewing things from a single point. an entire line can never be equidistant from a point, so the ends of the line will be thinner than the closest spot on that line relative to your eye.
i understand the current way has been working with very little complaint so far, but why settle for sufficient, when its a detail that is keeping us from accurate representations of 3d space. any environment will look slightly more realistic if properly rendered with slight fisheye.
Also the problems with post processing is, its done after perspective transform has already corrupted the render. so past a certain FOV, things becomming further behind the camera would be growing instead of shrinking...
could that be overcome in a vertex shader? is it possible to bypass the perspective transform, like maybe apply the vertex shader to an orthographic projection?
i would love to try this in UDK, but i dont think i have access to the matrix transforms. i could dust off c# to experiment, but id like to find a solution for UDK.
For what it's worth, the eye/brain can auto-correct a lot of things (sometimes incorrectly) and this would be something that I imaging it'd probably do it for.
That said, any issues with this regarding perspective and curves can be dealt with using a fisheye image effect set to an appropriate number.
The bigger problem is draw distance cutoff and Z-Depth based image effects. For an example look at Red Faction Guerilla - the mountains cut off far 'sooner' when you're looking straight at them, but turn a bit and they clip back on towards the side of the screen.
Some image effects (like Unity's Global Fog, I think) are actually done correctly, with trigonometry to determine correct distance from the camera.
an example i saw a while back. compare the inception reference to the 3d work. the movie reference looks more realistic, mostly because of the fish eye problem:
an example i saw a while back. compare the inception reference to the 3d work. the movie reference looks more realistic, mostly because of the fish eye problem
I don't know, I'm personally under the impression that the movie reference looked more realistic because it is actually real. Remove the fisheye and give it to the UDK scene (which is unfinished and has it's own weird/extreme FoV), and the movie would still look more realistic.
theres usually few reasons a real photo looks more realistic than a render, but this reason is the one that affects the silhouette and composition the most. what makes for a better composition, sweeping arcs cascading across your screen, or static straight boring lines? either answer can be done with a proper fish eye effect. if you want it to be realistic, a subtle fish eye is the way to go. similar to chromatic abberation, a light touch is all thats needed, although chromatic abberation is a camera lens thing, where as fish eye is inherent in all lenses, including the natural human eye
the thing to notice in those renders, through no fault of the artist at all, is the ceiling. that ceiling is farther away than the middle of the wall, since its above eye level. that means those vertical lines should be converging at a point directly over head, yet instead they are becoming farther apart as they travel up and away from you. that does not make sense...
i feel like a crazy person trying to point out the moon is made of cheese or somthing. if chewbacca lives on endor, you must aquit
but seriously, am i the only one that notices these things look curved in the real world? and to not curve them leads to an impossible paradox where distant things end up larger on screen than far things. imagine a world with negative distance, a world in which the farther things look larger than the close things... thats not a world id like to play in forever. but its the world all of our favorite games reside in.
its an overlooked error that needs to be sorted out at some point, and since we just got tesselation going, it seems like a good time to take care of this.
i imagine this will be a built in feature on all game engines, once a major engine like UDK adopts it at a fundamental level. you could just crank up the FOV without getting a headache from impossible geometries along the borders of the screen.
You can change vertex positions in shader in UDK, it's been in there for a few years now. So you shouldn't need to write your own shader or code
The job of the vertex shader is to convert from local or world coordinates to a screen space position. So this is exactly the right place to do it, too. You'd just want all your vertex shaders to not do a simple perspective divide, and you will probably barely notice the cost because it's only per vert.
You also need to consider the the eyes focal area when looking at a monitor. Its only few inches of "readable" surface area, so the eye moves around a lot. Simply mapping the screen to the accurate distortion and FoV of the eye will be incorrect because your eyes focusable fov is very small.
you are right. distortion is a natural behavior of any lens.
but most games are played on a, in comparison to the entire human field of view, small area. for this reason we wouldn't want to use a huge field of view to begin with, and in many cases the distortion here would be negligible. there's also the fact that this fish eye distortion would only be "correct" if we were looking at a curved screen. so both fish eye and rectilinear perspectives render WRONG or whatever.
ideally the screen should behave like a window, and for that you need the position of the eye relative to the screen. certainly, there would be some barrel distortion too, but pre-distorting would just mean the eye distorts it again. film has the same issue and for this reason there are rectilinear lenses. it's just that fish-eye as an effect is popular for the moment.
tl;dr: fish eye distortion isn't right. rectilinear isn't right. perspective breaks down if you don't look at the center of the screen. everything is wrong. everything renders wrong.
im sorry if im comming off as defensive. its only a theory im making up. i dont know any of this for sure, i just got a little carried away thinking about the idea. i enjoy thinking about this idea and i wanted to give it a proper explination. but at this point im kinda just repeating myself. the title of this thread was kinda dramatic, to draw attention, since i thought saying "perspective transformation matrix might be a little off" wouldnt get many views. id still love to see more discussion into this, and im gonna look into vertex shaders in udk. im not much of an environment artist, but i should probably put somthing together to do tests and see what looks better. thanks for all the replies so fast guys, im gonna keep thinking about this and try to show more than tell. cheers
you are right. distortion is a natural behavior of any lens.
tl;dr: fish eye distortion isn't right. rectilinear isn't right. perspective breaks down if you don't look at the center of the screen. everything is wrong. everything renders wrong.
very true. maybe we should solve that flat monitor problem first. it would be cool to have some kinda goggles that had hemispherical viewing screens, like 1 for each eye. you could get almost full peripheral vision that way. we are at a cool time in history, many problems have been solved, we know of a few more that are being worked on, and there is still so much to do in this field. better real time hair is probably more important than perspective distortions, since hair is usually at your focal point, and these distortions are worse along the edges.
Like EQ pointed out, we can imagine things for a while based on fancy ideas like this (and this thread is actually a very interesting read) ; but now that we established that indeed, stuff curves, I think it would be more productive to apply that to some more concrete examples than just cubes to see if there is any kind of benefit.
Even tho this effect is more "correct", that does't mean that games would automatically become better with it. For instance, imagine the case of a first person shooter. Panning left and right would most likely look very odd and disorienting if the image was curving fisheye-style, even slightly.
I don't think applying this to what we have now will make a big change, right now. But who knows, maybe in the future we will move away from simplistic models and use stuff like this. This is just one of many baby steps to take towards more realism - eventually it will all add up.
I'd be interested in seeing a demo of this in actual 3d. I typically feel nauseous with the extreme fisheye, like the old QT VR, or the flash example above. I think mostly it's the extreme curve that happens towards the edges of the screen that draw my eye away, something that a computer monitor can't handle without some sort of eye-tracking. Our natural reaction to want to look at something involves moving our view towards that, even if we have no intention of making a 'hard' change to our viewpoint. ie: saw something vaguely interesting in periphery while someone is talking; look at it then back to our speaker (it was a coffee stain on the tablecloth). This is a very fast motion where our view perspective is changed, but feels very minute and localized to the area of the coffee stain, mentally, we probably didn't shift much from looking at the speaker. As a viewer in a game, having a 'straight' view makes these small automatic eye movements is easy without moving the camera, or feeling that we need to.
You mention objects becoming larger past a certain point, and I'm not sure what you're talking about. Extending past the disappearing point?
Semi related, there's a local artist who does fantastic 6-point perspective paintings on spheres. He may be a good person to talk to about the subject.
You mention objects becoming larger past a certain point, and I'm not sure what you're talking about. Extending past the disappearing point?
what i meant was, when you draw in 3 point perspective, the 3 vanishing points form a triangle. every one of these lines is a horizon line, although only one is actually horizontal. the others would be horizon lines if the world your standing on lined up with one of the other 2 axis instead. so theres probably a better word than horizon line for these other 2 lines, but i havent heard about it if there is.
anyway, if you draw somthing past these horizon lines (in other words, outside that triangle), the object will be drawn incorrectly, because past that point, the object becomes further from you than its vanishing point, but in reality it has 2 vanishing points. in 6 point perspective, the lines that cross these horizons appear almost parrellel, which they should, since its similar to looking in 1 point perspective. which makes this transition smooth without distortion. all points are as far from each other as they are close to the viewer. or in other words, close things get bigger in 6 point perspective no matter how you draw it. it is a unifying theory of perspective. 3 point perspective breaks down in these areas, and isnt as accurate.
if you were able to infinitly zoom into a 6 point drawing, the lines you would see would become less and less curved, until at infinity, the whole image will be perfectly orthographic.
everyone here has tried making a character from photo orthographics at some point. if you take the picture from a kneeling position, you get a wide stomach and small head and feet. the further away you walked to take the photo, the more orthographic the photo became. if you were miles away taking a photo of somthing through a telescope, its basically orthographic. although until you reach infinity, its always curved, not orthographic, not straight lines.
it has to do with the ratio between your point of view, and the 2 objects whose size your comparing. if they are 3 feet apart and 100 miles away, they will almost look the same size. if they are 3 feet apart and 1 of them is 1 foot away from your eye, while the other is 2 feet away, it will be a big difference in size.
nothing closer than infinity looks truly orthographic, but it doesnt have to get far to seem pretty straight.
What you're referring to I think would only work well in games (where the viewer also has the ability to change their point of focus), and then I think only rarely (and if done subtly). As throttlekitty mentioned, our monitors don't detect eye movement - and neither does paper (magazines with screenshots, posters, etc).
If games rendered a fisheye effect that was any sort of noticeable, it would suck for screenshots and gameplay trailers. I have a hard enough time making stuff out when I'm looking at footage for upcoming games, I'd really hate to be screaming at my monitor, "WOULD YOU LOOK TO YOUR LEFT, DAMNIT!"
Additionally, there isn't a problem with 3 point perspective in drawings and other 2D images. Well, there's a "problem" with it being 100% accurate to life, but frankly - it has to be that way a lot of the time. It's used because it gives a (relatively) accurate and realistic perspective while doing two things - giving the artist control, while still giving the viewer freedom. It lets the artist pick a point of focus, an overall image to be viewed as a whole - while at the same time allowing the viewer to comfortably navigate the image with their own eyes. The artist in throttlekitty's post is a good example of what would happen otherwise - you can look at any specific point in the image and read it well, but the image as a whole is very distorted, it's difficult to take in.
tl;dr - Would it be cool sometimes? Sure. If a lot of other things that are not usually the case, are also the case.
i base absolutely every argument ive made on only 1 given rule that i believe to be closer to fact than gravity. that rule is that "close objects appear larger than they would if they were farther away", and i believe that rule applies to every point in space, in every direction. 3 point perspective is absolutely not even close to accurate, and i can consistantly break it as a theory. ive pointed out how to, just draw somthing outside that triangle. the farther the object goes from your point of view, it will GROW, instead of shrink. i believe that is always wrong, things gaining in distance should always shrink. if that assumption is sometimes wrong, than i would love to find out why. but im pretty certain its a universal truth that close things get bigger. that logic leads directly to curved lines, just compare distances and youll see. theres always a point thats closer to you than 2 other points on that line, those 2 other points are converging towards different vanishing points at all times. its just hard to notice when 1 point is always behind you. you cant see both points unless your FOV was above 180. but you can always see the affect those points have on the line describing the silohuette of the shapes, and it is always curved, at all distances and FOVs to some degree.
Three point perspective is a hack, obviously, but im not sure its one you know how to use? Each object can have its own vanishing point, silly.
I would like for you to explain why we should artificially simulate the FOV and lensing of the human eye on an object that is also going to be distorted by the human eye.
The thing you are describing is real, but it does not apply to 2d images in the way that you think it does. Perspective distortion is driven by your position, and the point you are focusing on. Your monitor is in 3d space. You sit in a chair a few feet from it -- you focus on a point in it. It's ludicrous to distort the image on the surface of the monitor to simulate some fixed perspective and particular level of distortion and then present it to a viewer whose eyes will be distorting it in an entirely different way.
What if the viewer focuses on something that isnt dead-center on their monitor? How would the monitor know the location of the viewer and distort the correct, realistic amount for the position it occupies in their field of view?
good read so far, thanks for the link. i was looking at their ellipses, and noticed they even point out there is a difference between constructing it with 2 points vs 1 an that neither are correct. they said the "radical" solution is to draw it with its major axis paralleled to the horizon line, which is like drawing it from 1 point perspective, as long as that point is directly above it. to be honest, the only reason that looks correct, is because its closest to how a 6 point perspective world would display it. 6 point perspective merges seamlessly between 1 2 and 3 points of perspective, and cannot cross into areas of "negative distance" like 1 2 and 3 points of perspective do. when your using 6 points correctly, every spot on the page is just as accurate as the next. and when you use it with a very small FOV, you will barely notice the difference from 3 point, other than its a little more correct around the 1 point perspective areas.
also with 6 point perspective, the monitor doesnt need to know where you are looking, it is accurate regardless, unlike 3 point perspective, which is only accurate directly between those 3 points, and fades in accuracy as your eye travels away from there.
Ok, I see what you're saying now. But we don't live on an infinite and flat planet, and that a given axis of the 3 point perspective lines should never actually meet and/or cross, since they are actually parallel.
However, when rendering on a flat surface with any given media, it is impossible to achieve, hence the term vanishing point, where we can't physically draw anything that small. Why should we draw something outside of the vanishing point if the medium won't allow it? (paint/graphite too thick, not enough pixels, etc)
Coming back around, Kurt has it right in that a post-effect shader would do the trick just nicely.
...you're not breaking anything as a theory. I'm not even sure what you mean by that. Do you mean stating something that people have known for a long time? The kind of thing you can read in almost any book on perspective? 3 point perspective isn't something you "break". It never claims to do the things you point out it doesn't do. It's a tool, a method. And like any method, it has its uses.
when your using 6 points correctly, every spot on the page is just as accurate as the next.
Yes. And the image as a whole looks whack as fuck. Which really kind of defeats the purpose when someone wants a nice, comfortable image to look at, or when you're trying to get a clear idea across.
It's pretty obvious you've given a lot of thought as to how things are "wrong". You need to start putting just as much thought in to why they're still used regardless.
Small edit: Most of the above is directed at your response towards 2D images, such as illustrations or concepts. As those are usually created with a purpose besides interactive immersion/realism in mind. I do think what you're talking about could have it's uses in a real-time 3D environment.
Three point perspective is a hack, obviously, but im not sure its one you know how to use? Each object can have its own vanishing point, silly.
I would like for you to explain why we should artificially simulate the FOV and lensing of the human eye on an object that is also going to be distorted by the human eye.
i understand 3 point perspective.
but every set of parallel lines actually has 2 vanishing points, only one of which you will see from any point of view. every object is composed of many lines, some are parallel. every set of parrallel lines converge to 2 opposite points both at an infinite distance from your eye, 180 degrees apart. if 2 sets of parallel lines are perpendicular, their vanishing points are always 90 degrees apart, all equally infinitly distant from your POV, and all aligned along a horizon line that is parralell with the plane these lines fall perpendicular within. a cube is the perfect way to show all attributes of a perspective system, since we are replicating 3 dimensions with perpendicular axis.
also we are already simulating the FOV of the human eye in all representations of perspective besides orthographic. the thing is. we arent doing it that well and its falling apart at the edges. there is a better way to project 3 dimensional points onto a 2d plane. so far throughought history, its been too expensive to use, not enough geometry, vertex transformations needed to use fast math like cos and sin rather than the sq roots that you need to calculate distance. and its not easy to draw on paper, most people dont look around enough to even notice the curves, and im sure theres other reasons. but we are at a point where this is possible, and through this discussion, we can discover if this hypothosis is accurate, then decide if its worth wasting system resources, then we find out if its a selling point or somthing people actually dislike for some uncanny valley reason. i dont know the answers to these questions, but ill try to come up with as many scenarios, examples, explinations i can, until someone can poke holes in it. then its time to experiment to see if its a noticable difference. at that point, maybe when i get a solution that is easy to drop your UDK level into and adjust the effect, ill give it out and we can all post some screenshots with before and after and figure out what looks better.
my only worry about UDK is that it keeps matrix transformations to itself as far as i know, so i would have to turn the FOV down to 5 or lower, pretend its orthographic, perform vertex transformations, and hope the slight distortions look better than the huge distortions of 3 point perspective.
the one thing we can all absolutely agree on, is that there is a setting, somewhere between perfectly straight lines, and lines that are so curvy that you cant understand the image. along that spectrum, what are the chances that magic number is "0" or "lacking any curvature?" my guess is that its close enough to 0 that its not much different from 3 point perspective, but far enough to fix any distortion caused by the fact that 3 point perspective doesn't account for situations where points on parallel lines become further from their vanishing point than the viewer. when points cross that line, they should be heading toward their opposite vanishing point. when you see proper 6 point perspective, and its done right, you wont think of it as fish eye perspective. you will think its rotoscoped from a photo. increasing FOV would do the same for the game's camera, as it would for a real camera.
...most people dont look around enough to even notice the curves
...well literally looking around for them is going to make you not notice the curves, haha. So that's probably one reason most people don't notice. I guess it's like anything else - most people don't focus on how light bounces off of surfaces, or the physics behind reflections, or various other things people in a visually-focused career might have to begin realizing.
Similar to how those things can really make a difference in a game's visuals, though, I guess it's possible this could as well. Even to people who don't really understand why.
If you want to check it out realtime I knocked up a fisheye shader for Max. It's got simple diffuse/spec/normal map support and lets you set how much of a fisheye distortion you want to apply.
If you want to show people how a subtle distortion can so drastically increase the realism of a scene, just do as Kurt Russell Fan Club suggests... who, IMHO has the second most awesome username here after Amsterdam Hilton Hotel.
If you want to check it out realtime I knocked up a fisheye shader for Max. It's got simple diffuse/spec/normal map support and lets you set how much of a fisheye distortion you want to apply.
also we are already simulating the FOV of the human eye in all representations of perspective besides orthographic. the thing is. we arent doing it that well and its falling apart at the edges. there is a better way to project 3 dimensional points onto a 2d plane. so far throughought history, its been too expensive to use, not enough geometry, vertex transformations needed to use fast math like cos and sin rather than the sq roots that you need to calculate distance. and its not easy to draw on paper, most people dont look around enough to even notice the curves, and im sure theres other reasons.
Well if your argument is that we aren't simulating "human" perspective, or the way humans process imaging, there's no doubt about that. However, until we are able crack how a human brain processes the data it receives from the eyeballs, that's kind of a hopeless dream. Believe it or not, the human brain DOES process the information and it doesn't just receive information, and that's something far beyond our current realm of understanding. That's why when you hang upside down or sideways, images aren't perceived as "turned", which can easily be proved by turning your head. Not really much of a difference, huh? It's probably also why humans have the ability to hallucinate, but the pseudo science here is endless and a bit unnecessary. I mean, if you really wanted to be true to life, cameras in games wouldn't even pick up anything but light, and everything would be a side effect of picking up light. Also, images would be processed upside down and then inverted to right side up after -- I mean I could go on, but it illustrates the point that the human vision doesn't "truly" need to be recreated.
The way I see it, the examples you have thus provided are merely a case of "different" shots, and I can't say for certain if that's completely impossible as my knowledge of 3d rendering isn't as vast as some others out there. However, with the proper shaders and adjustments, the inception example is more than doable (appears to use a wide angle lens basically, which is definitely possible with current technology. The user in question could have probably just turned up FOV quite a bit and the same types of distortions could occur with a few minor tweaks to post-processing)
Maybe it's just because I'm not much of a "let's emulate everything" type of guy, I feel the bigger issue is that games aren't as well directed as they could be when it comes to camera work. Films often change lenses for certain scenes, and often push aspects of scenes in order to focus the viewer's attention to specific aspects of the image. Not to mention there's theory out there that tries to shoe horn "standard" FOVs when, ideally, they should change depending on the type of effect you are going for, just like in films. Often times, people rely too much on "moving the camera" instead of merely changing the FOV, as those two concepts are vastly different from one another and can cause a large amount of difference when you're trying to recreate a specific scene.
In the end of the day, it doesn't REALLY matter if the rendering of a game engine isn't "accurate" as you really just need to treat it like a canvas, much like films treat it.
That's FOV 90 and warp weight (in the shader) of 0.25. The higher your field of view, the lower you should set that warp weight to (unless you want to decorate your monitor with your lunch)
Yeah, it's only realtime viewing. I don't know of any renderers that support realtime shaders
Yokai: its true that we will probably never fully understand the subjectivity of human perception, and the things that go on between seeing and understanding are beautifully confusing. ive always wondered if every person sees color differently. there are color blind people, so technically some people do see color different from others. so its kinda just a matter of how common those differences are. ive heard dogs see in grey scale, but i have no idea how we figured that out.
but im not saying we should emulate everything about the way people see, i just always noticed that there was something wrong with 3 point perspective when you plot points farther away from the middle. i notice the same thing in the way games render stuff and never really understood why. but then yesterday i was in udk, holding Z and letting go, which changes the FOV from 90 to 180 and back. (productive day ) noticing things getting larger when they should be getting smaller. i was thinking through the matrix transformations ive read in books, and realized that the perspective transforms didnt use square root and didnt solve for distance of anything. it just rotated and moved points relative to coordinate spaces and acted like the Z coordinate of the local space of the camera was the same as depth. thats when it hit me. a coordinate can go negative, depth/distance cannot. so then i thought maybe the absolute value of the zdepth would be close enough, but as i thought about it, i quickly realized it would create sharp angles in lines that are supposed to be parallel, which would look worse than 3 point perspective. so i came to the conclusion that it needs to use true depth from a single point, not just a direction. im still working out whether that conclusion is technically right, better in some way than the standard, practical, artistically pleasing, etc... but so far its been interesting to think about.
I remember being especially bothered by this playing the Total War games. Take a look a this screenshot; the building on the right shows nothing but contempt for how perspective actually works (and it's hard to find any screenshots of Rome Total War with buildings in it in which this isn't grating).
Edit: part of the problem may be that the FOV is rather high in games. 90 Degrees is apparently Quake's basic setting, which is a focal length of 16mm, which I think already classifies as an ultra wide angle lens - and its FOV can go up to 170 degrees.
How is your Max plugin doing the rounded fisheye? There's two methods I can think of:
-Render to a buffer and use a pixel shader to distort the image by delinearising the UVs
-Use a vertex shader to delinearise NDC so vertices bulge outwards slightly. As it pushes vertices out of the viewport not into it this should not cause problems with frustum culling.
Both methods have issues:
Method one has a memory + fillrate overhead and will likely lose some detail/have aliasing issues due to render/screen space no longer being 1:1.
Method two would have cracks in some models, requires well tessellated meshes to be believable, requires all vertex shaders used include the bulge code. Possibly not so much a problem on newer games due to tessellation keeping a good consistent triangles:pixels ratio.
Although it wouldn't exactly change gaming as we know it it has been a personal gripe of mine for a while too, and I see Omnicypher's original point that it breaks immersion at times. I think KRFC's shader seems like a step in the right direction though, it doesn't seem like something that could be fixed with a subtle fisheye post fx.
Can you show example of stuff actually getting bigger with distance though? I think I know what you mean but it's subtle, I haven't seen it happen in ways that actually fuck with my mind. I think it's visible in the doors and windows to the topleft corner of the image below, if you were to stack more doors/windows above those they'd look bigger and taller than the ones on ground level, which are closer to the camera, right?
How is your Max plugin doing the rounded fisheye? There's two methods I can think of:
-Render to a buffer and use a pixel shader to distort the image by delinearising the UVs
-Use a vertex shader to delinearise NDC so vertices bulge outwards slightly. As it pushes vertices out of the viewport not into it this should not cause problems with frustum culling.
Both methods have issues:
Method one has a memory + fillrate overhead and will likely lose some detail/have aliasing issues due to render/screen space no longer being 1:1.
Method two would have cracks in some models, requires well tessellated meshes to be believable, requires all vertex shaders used include the bulge code. Possibly not so much a problem on newer games due to tessellation keeping a good consistent triangles:pixels ratio.
method 1:
if your first step is to render to a buffer using a system that allows negative scaling along its periphery, you are not going to be able to fix that by moving pixels around. this isn't about getting nice looking round curves as a pretty effect, its about getting the points in space to translate correctly to a 2d plane to allow the lines they create to seamlessly transition between orientations in a unifying system for all distances and FOVs. the curves created by 6 point perspective are not screen space buldging effects, each curve moves through 3d space, and are more complicated in shape than that. it would be like trying to start with a 1 point perspective drawing and smudge it into a 3 point perspective drawing, it wont work.
method 2:
its more than just a bulge effect, it is actually moving points closer or farther away from the camera, based on whether the point is closer or father away from the camera. (it sounds redundant, but the technique makes up for the bit of math "distance from POV" which is left out of perspective transforms for old reasons .) the math behind this will never crack a mesh, as its not dependent on normal space translations. nothing is being displaced in a way that takes into account its surface normal, only its distance from the camera. that means the only "distortion" it would cause from the standard render method, would be far verts being farther away, and close verts being closer to the camera. but technically that is not distorting the image, its making it more accurate. far things should be thought of as farther away than close things. they should appear smaller, in all cases. with current methods, thats not always true, some things scale up with distance, which is why it is mathematically wrong.
but no matter what method you use, you cant fix things after you let the flawed perspective transform destroy your data with impossible negative scaling. it cant be done in post, it has to be done to orthographic points in 3D space, and it has to take into account distance from the camera, rather than distance from the axis perpendicular to the cameras direction. any other method allows for negative scaling, then trying to fix that, which to do you would probably have to multiply by the inverse of that perspective matrix transform, just to try to bring it back to orthographic so it can be done right.
this max shader is interesting, but ill have to play with it some more to see if its actually fixing the problem or just almost fixing it. is there a way i can turn this effect up more? like not limiting the number i can crank it up to? i want to see how things look with a more pronounced effect, to see if its behaving as i expect 6 point perspective would.
does this shader code replace the perspective transform and work directly on an orthographic projection? or is it just adding another transform on top of the original corrupted perspective projection? because without working from orthographic, i dont believe you can fix that negative scaling issue.
Replies
the effect exists regardless what lens you use, it happens any time an image is taken from a point. the only exception would be orthographic images, like from a scanner, which doesnt take the image from a point, but from many points on a plane.
I sometimes wonder if that's how most skateboarders think :P
Unless I'm talking outside of my arse, my eyes don't bend object to the extent that picture does.
If you want to add distance-based curvature then you can do it using a vertex shader or a post-process, and it'll give you exactly the effect you want. The vertex shader version will need you to have evenly balanced verts or else it'll fail to bend long objects, and the pixel shader will probably give a blurred effect. But if you're trying to make a game whose visual style relies on a curved look then it'd totally work.
im not suggesting games all become skateboarder handy cams, but fish eye is a natural effect that is always there. is not an optional effect you add, it is the natural reality of viewing things from a single point. an entire line can never be equidistant from a point, so the ends of the line will be thinner than the closest spot on that line relative to your eye.
i understand the current way has been working with very little complaint so far, but why settle for sufficient, when its a detail that is keeping us from accurate representations of 3d space. any environment will look slightly more realistic if properly rendered with slight fisheye.
Also the problems with post processing is, its done after perspective transform has already corrupted the render. so past a certain FOV, things becomming further behind the camera would be growing instead of shrinking...
could that be overcome in a vertex shader? is it possible to bypass the perspective transform, like maybe apply the vertex shader to an orthographic projection?
i would love to try this in UDK, but i dont think i have access to the matrix transforms. i could dust off c# to experiment, but id like to find a solution for UDK.
That said, any issues with this regarding perspective and curves can be dealt with using a fisheye image effect set to an appropriate number.
The bigger problem is draw distance cutoff and Z-Depth based image effects. For an example look at Red Faction Guerilla - the mountains cut off far 'sooner' when you're looking straight at them, but turn a bit and they clip back on towards the side of the screen.
Some image effects (like Unity's Global Fog, I think) are actually done correctly, with trigonometry to determine correct distance from the camera.
http://www.polycount.com/forum/showpost.php?p=1228911&postcount=1238
I don't know, I'm personally under the impression that the movie reference looked more realistic because it is actually real. Remove the fisheye and give it to the UDK scene (which is unfinished and has it's own weird/extreme FoV), and the movie would still look more realistic.
http://www.polycount.com/forum/showpost.php?p=1228911&postcount=1238
the thing to notice in those renders, through no fault of the artist at all, is the ceiling. that ceiling is farther away than the middle of the wall, since its above eye level. that means those vertical lines should be converging at a point directly over head, yet instead they are becoming farther apart as they travel up and away from you. that does not make sense...
i feel like a crazy person trying to point out the moon is made of cheese or somthing. if chewbacca lives on endor, you must aquit
but seriously, am i the only one that notices these things look curved in the real world? and to not curve them leads to an impossible paradox where distant things end up larger on screen than far things. imagine a world with negative distance, a world in which the farther things look larger than the close things... thats not a world id like to play in forever. but its the world all of our favorite games reside in.
its an overlooked error that needs to be sorted out at some point, and since we just got tesselation going, it seems like a good time to take care of this.
i imagine this will be a built in feature on all game engines, once a major engine like UDK adopts it at a fundamental level. you could just crank up the FOV without getting a headache from impossible geometries along the borders of the screen.
The job of the vertex shader is to convert from local or world coordinates to a screen space position. So this is exactly the right place to do it, too. You'd just want all your vertex shaders to not do a simple perspective divide, and you will probably barely notice the cost because it's only per vert.
but most games are played on a, in comparison to the entire human field of view, small area. for this reason we wouldn't want to use a huge field of view to begin with, and in many cases the distortion here would be negligible. there's also the fact that this fish eye distortion would only be "correct" if we were looking at a curved screen. so both fish eye and rectilinear perspectives render WRONG or whatever.
ideally the screen should behave like a window, and for that you need the position of the eye relative to the screen. certainly, there would be some barrel distortion too, but pre-distorting would just mean the eye distorts it again. film has the same issue and for this reason there are rectilinear lenses. it's just that fish-eye as an effect is popular for the moment.
tl;dr: fish eye distortion isn't right. rectilinear isn't right. perspective breaks down if you don't look at the center of the screen. everything is wrong. everything renders wrong.
very true. maybe we should solve that flat monitor problem first. it would be cool to have some kinda goggles that had hemispherical viewing screens, like 1 for each eye. you could get almost full peripheral vision that way. we are at a cool time in history, many problems have been solved, we know of a few more that are being worked on, and there is still so much to do in this field. better real time hair is probably more important than perspective distortions, since hair is usually at your focal point, and these distortions are worse along the edges.
Even tho this effect is more "correct", that does't mean that games would automatically become better with it. For instance, imagine the case of a first person shooter. Panning left and right would most likely look very odd and disorienting if the image was curving fisheye-style, even slightly.
I don't think applying this to what we have now will make a big change, right now. But who knows, maybe in the future we will move away from simplistic models and use stuff like this. This is just one of many baby steps to take towards more realism - eventually it will all add up.
You mention objects becoming larger past a certain point, and I'm not sure what you're talking about. Extending past the disappearing point?
Semi related, there's a local artist who does fantastic 6-point perspective paintings on spheres. He may be a good person to talk to about the subject.
edit to add video!
[ame="
what i meant was, when you draw in 3 point perspective, the 3 vanishing points form a triangle. every one of these lines is a horizon line, although only one is actually horizontal. the others would be horizon lines if the world your standing on lined up with one of the other 2 axis instead. so theres probably a better word than horizon line for these other 2 lines, but i havent heard about it if there is.
anyway, if you draw somthing past these horizon lines (in other words, outside that triangle), the object will be drawn incorrectly, because past that point, the object becomes further from you than its vanishing point, but in reality it has 2 vanishing points. in 6 point perspective, the lines that cross these horizons appear almost parrellel, which they should, since its similar to looking in 1 point perspective. which makes this transition smooth without distortion. all points are as far from each other as they are close to the viewer. or in other words, close things get bigger in 6 point perspective no matter how you draw it. it is a unifying theory of perspective. 3 point perspective breaks down in these areas, and isnt as accurate.
if you were able to infinitly zoom into a 6 point drawing, the lines you would see would become less and less curved, until at infinity, the whole image will be perfectly orthographic.
everyone here has tried making a character from photo orthographics at some point. if you take the picture from a kneeling position, you get a wide stomach and small head and feet. the further away you walked to take the photo, the more orthographic the photo became. if you were miles away taking a photo of somthing through a telescope, its basically orthographic. although until you reach infinity, its always curved, not orthographic, not straight lines.
it has to do with the ratio between your point of view, and the 2 objects whose size your comparing. if they are 3 feet apart and 100 miles away, they will almost look the same size. if they are 3 feet apart and 1 of them is 1 foot away from your eye, while the other is 2 feet away, it will be a big difference in size.
nothing closer than infinity looks truly orthographic, but it doesnt have to get far to seem pretty straight.
If games rendered a fisheye effect that was any sort of noticeable, it would suck for screenshots and gameplay trailers. I have a hard enough time making stuff out when I'm looking at footage for upcoming games, I'd really hate to be screaming at my monitor, "WOULD YOU LOOK TO YOUR LEFT, DAMNIT!"
Additionally, there isn't a problem with 3 point perspective in drawings and other 2D images. Well, there's a "problem" with it being 100% accurate to life, but frankly - it has to be that way a lot of the time. It's used because it gives a (relatively) accurate and realistic perspective while doing two things - giving the artist control, while still giving the viewer freedom. It lets the artist pick a point of focus, an overall image to be viewed as a whole - while at the same time allowing the viewer to comfortably navigate the image with their own eyes. The artist in throttlekitty's post is a good example of what would happen otherwise - you can look at any specific point in the image and read it well, but the image as a whole is very distorted, it's difficult to take in.
tl;dr - Would it be cool sometimes? Sure. If a lot of other things that are not usually the case, are also the case.
There is a bit of curve to that picture, like 5 pixels worth of curve on a full screen version.
http://www.handprint.com/HP/WCL/perspect5.html#curvilinear
I would like for you to explain why we should artificially simulate the FOV and lensing of the human eye on an object that is also going to be distorted by the human eye.
The thing you are describing is real, but it does not apply to 2d images in the way that you think it does. Perspective distortion is driven by your position, and the point you are focusing on. Your monitor is in 3d space. You sit in a chair a few feet from it -- you focus on a point in it. It's ludicrous to distort the image on the surface of the monitor to simulate some fixed perspective and particular level of distortion and then present it to a viewer whose eyes will be distorting it in an entirely different way.
What if the viewer focuses on something that isnt dead-center on their monitor? How would the monitor know the location of the viewer and distort the correct, realistic amount for the position it occupies in their field of view?
good read so far, thanks for the link. i was looking at their ellipses, and noticed they even point out there is a difference between constructing it with 2 points vs 1 an that neither are correct. they said the "radical" solution is to draw it with its major axis paralleled to the horizon line, which is like drawing it from 1 point perspective, as long as that point is directly above it. to be honest, the only reason that looks correct, is because its closest to how a 6 point perspective world would display it. 6 point perspective merges seamlessly between 1 2 and 3 points of perspective, and cannot cross into areas of "negative distance" like 1 2 and 3 points of perspective do. when your using 6 points correctly, every spot on the page is just as accurate as the next. and when you use it with a very small FOV, you will barely notice the difference from 3 point, other than its a little more correct around the 1 point perspective areas.
also with 6 point perspective, the monitor doesnt need to know where you are looking, it is accurate regardless, unlike 3 point perspective, which is only accurate directly between those 3 points, and fades in accuracy as your eye travels away from there.
However, when rendering on a flat surface with any given media, it is impossible to achieve, hence the term vanishing point, where we can't physically draw anything that small. Why should we draw something outside of the vanishing point if the medium won't allow it? (paint/graphite too thick, not enough pixels, etc)
Coming back around, Kurt has it right in that a post-effect shader would do the trick just nicely.
Yes. And the image as a whole looks whack as fuck. Which really kind of defeats the purpose when someone wants a nice, comfortable image to look at, or when you're trying to get a clear idea across.
It's pretty obvious you've given a lot of thought as to how things are "wrong". You need to start putting just as much thought in to why they're still used regardless.
Small edit: Most of the above is directed at your response towards 2D images, such as illustrations or concepts. As those are usually created with a purpose besides interactive immersion/realism in mind. I do think what you're talking about could have it's uses in a real-time 3D environment.
i understand 3 point perspective.
but every set of parallel lines actually has 2 vanishing points, only one of which you will see from any point of view. every object is composed of many lines, some are parallel. every set of parrallel lines converge to 2 opposite points both at an infinite distance from your eye, 180 degrees apart. if 2 sets of parallel lines are perpendicular, their vanishing points are always 90 degrees apart, all equally infinitly distant from your POV, and all aligned along a horizon line that is parralell with the plane these lines fall perpendicular within. a cube is the perfect way to show all attributes of a perspective system, since we are replicating 3 dimensions with perpendicular axis.
also we are already simulating the FOV of the human eye in all representations of perspective besides orthographic. the thing is. we arent doing it that well and its falling apart at the edges. there is a better way to project 3 dimensional points onto a 2d plane. so far throughought history, its been too expensive to use, not enough geometry, vertex transformations needed to use fast math like cos and sin rather than the sq roots that you need to calculate distance. and its not easy to draw on paper, most people dont look around enough to even notice the curves, and im sure theres other reasons. but we are at a point where this is possible, and through this discussion, we can discover if this hypothosis is accurate, then decide if its worth wasting system resources, then we find out if its a selling point or somthing people actually dislike for some uncanny valley reason. i dont know the answers to these questions, but ill try to come up with as many scenarios, examples, explinations i can, until someone can poke holes in it. then its time to experiment to see if its a noticable difference. at that point, maybe when i get a solution that is easy to drop your UDK level into and adjust the effect, ill give it out and we can all post some screenshots with before and after and figure out what looks better.
my only worry about UDK is that it keeps matrix transformations to itself as far as i know, so i would have to turn the FOV down to 5 or lower, pretend its orthographic, perform vertex transformations, and hope the slight distortions look better than the huge distortions of 3 point perspective.
the one thing we can all absolutely agree on, is that there is a setting, somewhere between perfectly straight lines, and lines that are so curvy that you cant understand the image. along that spectrum, what are the chances that magic number is "0" or "lacking any curvature?" my guess is that its close enough to 0 that its not much different from 3 point perspective, but far enough to fix any distortion caused by the fact that 3 point perspective doesn't account for situations where points on parallel lines become further from their vanishing point than the viewer. when points cross that line, they should be heading toward their opposite vanishing point. when you see proper 6 point perspective, and its done right, you wont think of it as fish eye perspective. you will think its rotoscoped from a photo. increasing FOV would do the same for the game's camera, as it would for a real camera.
...well literally looking around for them is going to make you not notice the curves, haha. So that's probably one reason most people don't notice. I guess it's like anything else - most people don't focus on how light bounces off of surfaces, or the physics behind reflections, or various other things people in a visually-focused career might have to begin realizing.
Similar to how those things can really make a difference in a game's visuals, though, I guess it's possible this could as well. Even to people who don't really understand why.
Grab it here:
http://pastebin.com/rUczynYh
Very vomit inducing at high levels.
N'AWWWWWW :poly142:
sweet! thanks dude. i dont know how to install it though.. how do i put the text file into a custom shader in max? is this file a .fx?
- save to a new text file called fisheye.fx
Do this:
- Apply the material to your mesh and change the settings to whatever you want.
Well if your argument is that we aren't simulating "human" perspective, or the way humans process imaging, there's no doubt about that. However, until we are able crack how a human brain processes the data it receives from the eyeballs, that's kind of a hopeless dream. Believe it or not, the human brain DOES process the information and it doesn't just receive information, and that's something far beyond our current realm of understanding. That's why when you hang upside down or sideways, images aren't perceived as "turned", which can easily be proved by turning your head. Not really much of a difference, huh? It's probably also why humans have the ability to hallucinate, but the pseudo science here is endless and a bit unnecessary. I mean, if you really wanted to be true to life, cameras in games wouldn't even pick up anything but light, and everything would be a side effect of picking up light. Also, images would be processed upside down and then inverted to right side up after -- I mean I could go on, but it illustrates the point that the human vision doesn't "truly" need to be recreated.
The way I see it, the examples you have thus provided are merely a case of "different" shots, and I can't say for certain if that's completely impossible as my knowledge of 3d rendering isn't as vast as some others out there. However, with the proper shaders and adjustments, the inception example is more than doable (appears to use a wide angle lens basically, which is definitely possible with current technology. The user in question could have probably just turned up FOV quite a bit and the same types of distortions could occur with a few minor tweaks to post-processing)
Maybe it's just because I'm not much of a "let's emulate everything" type of guy, I feel the bigger issue is that games aren't as well directed as they could be when it comes to camera work. Films often change lenses for certain scenes, and often push aspects of scenes in order to focus the viewer's attention to specific aspects of the image. Not to mention there's theory out there that tries to shoe horn "standard" FOVs when, ideally, they should change depending on the type of effect you are going for, just like in films. Often times, people rely too much on "moving the camera" instead of merely changing the FOV, as those two concepts are vastly different from one another and can cause a large amount of difference when you're trying to recreate a specific scene.
In the end of the day, it doesn't REALLY matter if the rendering of a game engine isn't "accurate" as you really just need to treat it like a canvas, much like films treat it.
Customize -> Preferences -> Viewports tab -> Choose Driver -> Direct3d
Then restart.
Should look like this:
That's FOV 90 and warp weight (in the shader) of 0.25. The higher your field of view, the lower you should set that warp weight to (unless you want to decorate your monitor with your lunch)
Yeah, it's only realtime viewing. I don't know of any renderers that support realtime shaders
but im not saying we should emulate everything about the way people see, i just always noticed that there was something wrong with 3 point perspective when you plot points farther away from the middle. i notice the same thing in the way games render stuff and never really understood why. but then yesterday i was in udk, holding Z and letting go, which changes the FOV from 90 to 180 and back. (productive day
Edit: part of the problem may be that the FOV is rather high in games. 90 Degrees is apparently Quake's basic setting, which is a focal length of 16mm, which I think already classifies as an ultra wide angle lens - and its FOV can go up to 170 degrees.
-Render to a buffer and use a pixel shader to distort the image by delinearising the UVs
-Use a vertex shader to delinearise NDC so vertices bulge outwards slightly. As it pushes vertices out of the viewport not into it this should not cause problems with frustum culling.
Both methods have issues:
Method one has a memory + fillrate overhead and will likely lose some detail/have aliasing issues due to render/screen space no longer being 1:1.
Method two would have cracks in some models, requires well tessellated meshes to be believable, requires all vertex shaders used include the bulge code. Possibly not so much a problem on newer games due to tessellation keeping a good consistent triangles:pixels ratio.
Can you show example of stuff actually getting bigger with distance though? I think I know what you mean but it's subtle, I haven't seen it happen in ways that actually fuck with my mind. I think it's visible in the doors and windows to the topleft corner of the image below, if you were to stack more doors/windows above those they'd look bigger and taller than the ones on ground level, which are closer to the camera, right?
method 1:
if your first step is to render to a buffer using a system that allows negative scaling along its periphery, you are not going to be able to fix that by moving pixels around. this isn't about getting nice looking round curves as a pretty effect, its about getting the points in space to translate correctly to a 2d plane to allow the lines they create to seamlessly transition between orientations in a unifying system for all distances and FOVs. the curves created by 6 point perspective are not screen space buldging effects, each curve moves through 3d space, and are more complicated in shape than that. it would be like trying to start with a 1 point perspective drawing and smudge it into a 3 point perspective drawing, it wont work.
method 2:
its more than just a bulge effect, it is actually moving points closer or farther away from the camera, based on whether the point is closer or father away from the camera. (it sounds redundant, but the technique makes up for the bit of math "distance from POV" which is left out of perspective transforms for old reasons .) the math behind this will never crack a mesh, as its not dependent on normal space translations. nothing is being displaced in a way that takes into account its surface normal, only its distance from the camera. that means the only "distortion" it would cause from the standard render method, would be far verts being farther away, and close verts being closer to the camera. but technically that is not distorting the image, its making it more accurate. far things should be thought of as farther away than close things. they should appear smaller, in all cases. with current methods, thats not always true, some things scale up with distance, which is why it is mathematically wrong.
but no matter what method you use, you cant fix things after you let the flawed perspective transform destroy your data with impossible negative scaling. it cant be done in post, it has to be done to orthographic points in 3D space, and it has to take into account distance from the camera, rather than distance from the axis perpendicular to the cameras direction. any other method allows for negative scaling, then trying to fix that, which to do you would probably have to multiply by the inverse of that perspective matrix transform, just to try to bring it back to orthographic so it can be done right.
does this shader code replace the perspective transform and work directly on an orthographic projection? or is it just adding another transform on top of the original corrupted perspective projection? because without working from orthographic, i dont believe you can fix that negative scaling issue.
It would not be possible to do this smoothly without a lot of tesselation...
For someone that can program in 10 languages this should not, of course, be a problem.