Enable rendering with unbounded far distance

[Flarkk]

This PR allows rendering with unlimited camera zfar - or rather limited by the sole floating point range i.e. ~3.4e+38 in single precision and ~1.8e+308 with doubles.

This effectively untaps the potential of reverse-z depth buffer especially with single precision build.

Outline

Currently zfar cannot be pushed further than barely ~1e6 times znear (with single precision floats) because of numerical precision limitations with some methods of struct Projection.
With the default znear = 0.05 this sets the maximum view distance to ~50 km.
While it's enough in most cases, it can be a hard limitation for open worlds, space games, and any very large scene.
Beyond this limit, scene rendering currently breaks and Godot starts spaming errors.

This PR removes this limitation with 3 key changes :

• Stores the 6 frustum planes alongside the projection matrix in the rendering server's internals. Both are equivalent representations except that the former allows retrieving near/far distances and boundary points with much more accurate numerical precision
• Restricts the use of the matrix representation to the sole cases where projections / un-projections are performed. Use the 6 planes in any other situation (typically to retrieve znear / zfar, get boundary points, etc...)
• Bundles the 6 planes representation into a new struct Frustum with a few helper methods for convenience. This avoids relying of Vector<Plane> everywhere and improves readability

This PR focuses on in-game rendering and camera preview in Editor.
On top of it, PR #100896 allows editing very distant objects right in Godot's editor

TODOs (now complete)

• Z-fighting with very far away opaque objects (incl. Sky) in Compatibility and Mobile : Intended behavior (see caveats)
• Objects farther than 1e20 disappear when Occlusion culling is on : Solved by Switch occlusion culling to be based on depth instead of Euclidean distance #103798 (see Enable rendering with unbounded far distance #99986 (comment))

Demo : full-scale scene

large_zfar.zip

Vegetation is ~1m tall and ~10m from the observer
The terrain is 10 km large
The moon's radius is 1,700 km and is 400,000 km from the observer
The Andromeda galaxy is 152,000 light-years wide and is 2.5 million light years from the observer

Performance

This PR improves very slightly the frame process time of an empty scene by ~ 0.5%.
This is likely due to the significant number of planes retrieval operations avoided by the fact the 6 planes are already made available.

Before	After

Effects compatibility

I spent a considerable amount of time making sure all effects work with large zfar.
Some require additional PRs to be merged.
Test project for effects : effects.zip (make sure to preview the large zfar camera in each scene)

Effects	Works with large zfar
Bokeh DOF	✔️ with #99755
SSR	✔️
SSIL	✔️
SSAO	✔️
Sky	✔️
Shadows	✔️
SDFGI	✔️ with #104120
SSS	✔️ with #99755
Fog	✔️

Caveats

• XR is left unchanged for now because this would require the XR APIs to return frustum planes in addition to the projection matrix. This may be the subject of another PR
• Unlike Forward+, Mobile and Compatibility renderers use 24bit UNORM depth buffers which don't get any benefit from reverse-z, so z-fighting will happen near the far plane when zfar is very large. This can notably cause the sky to occlude very distant objects.

Optional dependencies

#99961 and #99962 may help ensuring non-regression on Viewport and Camera3D.

---

Documentation twin PR : godotengine/godot-docs#11776
Closes godotengine/godot-proposals#10515
Closes #55070
Supersedes #95944

[Flarkk]

Rebased and fixed style issues to allow CI to run.

[Flarkk]

Rebased on top of #99974

[Calinou]

Objects farther than 1e20 disappear when Occlusion culling is on

This can probably be worked around by assuming the object is visible when it's further away from the camera than this distance (and skipping any occlusion culling checks).

Embree doesn't support double precision, so precision=double builds can't do any better here.

[Flarkk]

Haven't got back to this PR yet, but I think it's likely caused by an overflow with normalize() or length() somewhere.

1e19 ~ 1e20 is the threshold value that exceeds the maximum representable float value when squared (~1e38).

If this overflow happens in Embree then yes, we don't have any other choice but trying to work around it. Will take a look at your suggestion once I complete the work on another incoming related PR : uncapping zfar in the editor itself.

[Flarkk]

I got the culprit : indeed a call to length() that overflows on very far away geometry :

godot/servers/rendering/renderer_scene_occlusion_cull.h

Line 75 in efa1443

float min_depth = (closest_point - p_cam_position).length();

Just issued #100907 that solves the problem, and enables proper occlusion culling even on very far away objects.

---

Edit : turns out that this fix inadvertently reverts another fix. I'm figuring out alternatives
Edit 2 : this is solved by #103798

[Monniasza]

Is this issue being worked on? I'm trying to make a spaceflight simulation game that has objects both near and far.

[Flarkk]

Yes it is, although I couldn't spare time the past weeks.

Note that if the farthest objects in your scene are under ~1e19m close to the camera you will not encounter any occlusion culling issue with this PR. Also you can still exclude individual objects from occlusion culling processing in case they're farther than that.

[Flarkk]

Thanks @Ansraer
Squashed commits.

Now I'm going to figure out what changes may be needed to the docs and open a PR if any.
Will link it here.

[Flarkk]

Just spent a couple of hours seeking documentation sections that would deserve an update. I didn't find anything major but still suggested minor changes in godotengine/godot-docs#11776

[Calinou]

Can you rebase this PR on master? I'd like to take a (hopefully final) look.

[Flarkk]

Rebased.

[Calinou]

Tested locally on 3d_primitives.tscn from the Tonemapping and Color Correction demo, it works as expected. Code looks good to me. Thanks for adding unit tests too 🙂

Some feedback:

• I got a Vulkan driver crash when setting a Camera3D node's Far property to 9999999 even though I wasn't currently previewing it:

================================================================
handle_crash: Program crashed with signal 11
Engine version: Godot Engine v4.7.dev.custom_build (7b9f27a17049c8d16ff1492cd2407802e53756b6)
Dumping the backtrace. Please include this when reporting the bug on: https://github.com/godotengine/godot/issues
[1] /lib64/libc.so.6(+0x1a290) [0x7fcc83c92290] (??:0)
[2] /lib64/libnvidia-glcore.so.580.119.02(+0xddd330) [0x7fcbed9dd330] (??:0)
[3] bin/godot.linuxbsd.editor.x86_64() [0x6629df2] (/home/hugo/Documents/Git/godotengine/godot/drivers/vulkan/rendering_device_driver_vulkan.cpp:6855)
[4] bin/godot.linuxbsd.editor.x86_64() [0x662ad97] (/home/hugo/Documents/Git/godotengine/godot/drivers/vulkan/rendering_device_driver_vulkan.cpp:7043)
[5] bin/godot.linuxbsd.editor.x86_64() [0x6613465] (/home/hugo/Documents/Git/godotengine/godot/drivers/vulkan/rendering_device_driver_vulkan.cpp:3176)
[6] bin/godot.linuxbsd.editor.x86_64() [0xa0b7e30] (/home/hugo/Documents/Git/godotengine/godot/servers/rendering/rendering_device.cpp:6580 (discriminator 5))
[7] bin/godot.linuxbsd.editor.x86_64() [0xa0b8d53] (/home/hugo/Documents/Git/godotengine/godot/servers/rendering/rendering_device.cpp:6677)
[8] bin/godot.linuxbsd.editor.x86_64() [0xa0ccffb] (/home/hugo/Documents/Git/godotengine/godot/servers/rendering/rendering_device_driver.h:133)
[9] bin/godot.linuxbsd.editor.x86_64() [0xa0c9a91] (/home/hugo/Documents/Git/godotengine/godot/servers/rendering/rendering_device.cpp:7153)
[10] bin/godot.linuxbsd.editor.x86_64() [0xa518783] (/home/hugo/Documents/Git/godotengine/godot/servers/rendering/renderer_rd/renderer_compositor_rd.cpp:134)
[11] bin/godot.linuxbsd.editor.x86_64() [0xa2162c1] (/home/hugo/Documents/Git/godotengine/godot/servers/rendering/rendering_server_default.cpp:112)
[12] bin/godot.linuxbsd.editor.x86_64() [0xa218c59] (/home/hugo/Documents/Git/godotengine/godot/servers/rendering/rendering_server_default.cpp:447)
[13] bin/godot.linuxbsd.editor.x86_64() [0x411d526] (/home/hugo/Documents/Git/godotengine/godot/main/main.cpp:4972)
[14] bin/godot.linuxbsd.editor.x86_64() [0x40073e0] (/home/hugo/Documents/Git/godotengine/godot/platform/linuxbsd/os_linuxbsd.cpp:995 (discriminator 1))
[15] bin/godot.linuxbsd.editor.x86_64() [0x401fcc5] (/home/hugo/Documents/Git/godotengine/godot/platform/linuxbsd/godot_linuxbsd.cpp:121)
[16] /lib64/libc.so.6(+0x35b5) [0x7fcc83c7b5b5] (??:0)
[17] /lib64/libc.so.6(__libc_start_main+0x88) [0x7fcc83c7b668] (??:0)
[18] bin/godot.linuxbsd.editor.x86_64() [0x3ffb825] (??:?)
-- END OF C++ BACKTRACE --
================================================================

I can't reproduce this crash consistently though. I've been able to set a Far property to 2**63 - 1 without trouble:

• In Forward+, I noticed that perspective with a FOV of 1 degree manages better precision than orthographic rendering:

perspective_vs_orthographic.mp4

This is also noticeable when SSR is enabled (not shown on the video above). It's strange, as I would logically expect the opposite (very low perspective FOVs often exacerbate depth buffer precision issues).

It's the opposite in Mobile where orthogonal has very good precision, but perspective looks very unstable with lots of Z-fighting.

[Flarkk]

Thanks for adding unit tests too 🙂

You might also be interested in reviewing #99961 and #99962 that add even more tests to Viewport and Camera3D. I heavily relied on them when crafting this PR.

I kept them separate for the sake of simplicity but it could be beneficial to get them merged in the same batch.

[Calinou]

@Calinou If you want to try reproducing the crash again, use a dev build so that the stack trace will have debugging symbols, by compiling Godot with dev_build=yes dev_mode=yes (I forget which is which so I just always use both).

The Godot backtrace has debug symbols (I always use debug_symbols=yes), but we can't get a backtrace from within the NVIDIA driver as it's proprietary and we don't have symbols for it.

[Flarkk]

In Forward+, I noticed that perspective with a FOV of 1 degree manages better precision than orthographic rendering

Does it specifically happen with large zfar or is it a more general observation ? Could you notice the same behavior on master ?

[Calinou]

Does it specifically happen with large zfar or is it a more general observation ? Could you notice the same behavior on master ?

The difference is only visible with large Z far (it's not noticeable when set to 10000 or below). I can reproduce this on master too.

[Flarkk]

I got a Vulkan driver crash when setting a Camera3D node's Far property to 9999999

That's a weird one.
I'd have bet on NaNs in some shader code (SSR maybe as it was rewritten since I opened this PR) like in #104120, if you had previewed the camera, but you hadn't.

That would help if you could reproduce the issue.

[BastiaanOlij]

Ok, just some initial feedback coming in very late as I have not followed the development of this. So take a few things with a grain of salt.

First, just administrative, I've just added a bunch of cleanup code that properly handles un-projection and removes some of the workarounds added for the frustum camera node (see #116752). As you're renaming some of these to move them out of the way and introduce a new variable for a new purpose with the old name, that's going to cause some grief :) I'm hoping my PR gets merged before yours as the cleanup on your end is probably easier.

The other point of immediate concern is this clashing with the work I'm doing in #116424 which is a long outstanding change to allow more flexibility on the projection matrices provided to rendering and specifically the ability to supply more than one for stereo rendering. The long term goal is to simplify things even more now that we've properly exposed the Projection matrix class to the engine.

In that I'm concerned that XR has not been taken into account here because there is a good chance that things need to be solved differently for stereo rendering to work properly with this. Especially if we're going to work from frustum planes forward instead of from a projection matrix backwards, we have a problem.

In that I'd definitely like to know more about why it's important to have the frustum planes, I'm assuming there is much more going on here than just needing to know the original near and far distances. That could be solved in much easier ways.

[Flarkk]

I'm hoping my PR gets merged before yours as the cleanup on your end is probably easier.

Agree !

In that I'd definitely like to know more about why it's important to have the frustum planes, I'm assuming there is much more going on here than just needing to know the original near and far distances. That could be solved in much easier ways.

The core issue with the z-far value not being retrievable from an infinite perspective matrix can technically be resolved by simply knowing the original far distance.

There are however a few other considerations that motivated this implementation rather than simply passing z-far around:

• orthographic matrices can also degenerate across X and Y axis when right/left or top/bottom planes are very far apart (typical use case : rendering a star map at scale). This implementation solves this case too
• several Projection´s getters - besides get_z_far() - also extract the far distance from the matrix (e.g. half extents, corners, and a few others). Without a rewrite, they would need to accept a clunky z_far_override argument
• the implementation should, wherever possible, enforce the use of precision-proof functions to prevent regressions. In this context, replacing Projection with Frustum objects seemed like a more robust pattern than simply adding a z_far argument alongside Projection in multiple functions
• since the rendering server already computes the 6 projection planes from the matrix, providing them upfront doesn’t introduce any harm

This PR is more about enabling flexibility to work with both planes and matrices as needed, rather than adopting a frustum-plane-first approach.

We could perhaps keep that flexibility in #116424 by allowing RenderingServer.camera_set_projections() to take an optional array of frustums that would override the 6 planes extracted by default from the matrices. Camera3D would set it unless user provides a custom matrix. XRCamera3D might be able to set it too depending of how we get frustum / projection data from the hmd.

[BastiaanOlij]

Gotcha, I can see the validity of that reasoning. Indeed with planes you would potentially provide a bunch of accuracy issues. But it's an awfully invasive change that may have very unpredictable consequences.

Purely for XR:

• The biggest difficulty is that we don't control the source of the projection matrices. OpenXR likely is ok, we simply get left/right/top/bottom half-fovs to construct an asymmetrical projection matrix from, we can probably come up with logic that instead outputs planes. Other systems however is hard to predict, there are those that simply give us the projection matrix they want us to use, so we'd go from projection matrix to planes back to projection matrix.
• The main projection is repurposed to create an encompassing frustum. This so culling is optimised and so we can do reprojection of effects that don't process stereo. Now here your plane approach may end up being a bonus, because we go back to the planes (but offset on IPD and possibly rotated when slanted displays are used) to construct this encompassing volume.
• We submit depth buffer data back to the XR environment so the headset can do internal reprojection to counter minute head movements between position sampling and display. It is currently hard to predict how this will behave if infinite z is used.

We need to get XR right because I know several people (including myself) who are already working on space based games and would jump at using this feature as we're currently using all sorts of workarounds. If it immediately breaks on XR we have a problem.

None XR concerns:

My major one would be that supplying planes to the system allows far more freedom than we may be comfortable with or can predict the outcome off. I already ran into a lot of issues around this with my projection PR and added a few checks to ensure our projection matrices confirm to what our rendering engine can handle.

Originally I wanted to embed the offsets into the projection matrix at the source (we do end up doing this once we start rendering) but it made it impossible to correctly render things as many rendering effects require us to know the offset that was applied.

Also our rendering engine requires the near and far plane to be perpendicular to the z direction and parallel to each other.
Similarly there are limits that need to be imposed to the top, bottom, left and right planes.
Getting these right, well documented, and also not so strict that we deny real use cases, is going to be something we need to do.

[Flarkk]

Got it.

How do you see the next steps on these PRs ?
Merge yours first once ready, then figure out how this one can be adapted ? Or merge this one whenever all concerns get answered, and implement plane support in yours ? Or something else ?

I'll start digging into your remarks. Note that I'm able to test on a HMD (I have a Vive XR Elite) but I'm pretty new at it with Godot so might need some guidance.

[Flarkk]

A bunch of early feedbacks and questions on the points you raised:

Other systems however is hard to predict, there are those that simply give us the projection matrix they want us to use, so we'd go from projection matrix to planes back to projection matrix

1- Can we safely assume that near and far distances these matrices are built from are technical values that can be ignored (replaced by the ones we want to render with)? We could easily extract the 6 lateral planes then reconstruct the near and far planes from our values.

2- There is no back and forth: rendering server already uses both matrices (for rendering) and planes (for culling). This PR just provides both of them upfront instead of deriving one from the other. When I wrote earlier "replacing Projection by Frustum objects" I meant in the internal logic, not in rendering server's interface.

We submit depth buffer data back to the XR environment so the headset can do internal reprojection to counter minute head movements between position sampling and display. It is currently hard to predict how this will behave if infinite z is used.

This one deserves testing definitely. In theory nothing prevents reprojecting from infinite matrices, but I don't know much about how headsets handle it. If there is some documentation provided by several manufacturers, it will certainly help me out getting a sense of it.

I know several people [...] who [...] would jump at using this feature as we're currently using all sorts of workarounds. If it immediately breaks on XR we have a problem.

Definitely agree

My major one would be that supplying planes to the system allows far more freedom than we may be comfortable with or can predict the outcome off

We do have to handle this properly indeed, both with accurate documentation and input validation.
It shouldn't be too strict though:

• planes override would only be exposed at server API level for advanced use cases
• it may be ok if planes are not strictly synchronized to the matrix in some cases (e.g. transitional visual effects based on a custom matrix)
• considering planes are mainly used for culling, and distances / corners extraction, non-standard or non-synchronized frustums wouldn't break rendering anyway

[Flarkk]

Rebased

[Flarkk]

@BastiaanOlij and I had a meeting to discuss the coordination between this PR and #116424. Here are the takeaways:

• Implement multi-layer projection camera #116424 implements key changes inside the rendering server to support custom projections per eye/view/layer, which this PR will need to be rebased onto. It looks simpler to get Implement multi-layer projection camera #116424 merged first. With this PR on top we would get both custom projections support at server api level for all eyes, and unbounded z-far but only when non-custom projections are used (i.e. perspective, orthographic or frustum).

• next goal after Implement multi-layer projection camera #116424 is to move the projection matrices' building code from the rendering server to camera nodes, and remove the setters for perspective, ortho and frustum modes from the rendering server api (only set_projections remains). Supporting large z-far at this point requires that set_projections takes an array of planes in addition to the arrays of projections and transforms. This enables the planes to be build in the camera nodes too, then passed to the server.
  There will be a bunch of design decisions to make at this point:

  • should we enforce that matrices and planes are in sync? Are there use cases where the user wants to cull against a different frustum than the one used for projecting? (Bastiaan mentioned that @lyuma may have one in mind)
  • should we check/enforce that planes form a valid frustum?
  • should planes be optional (retrieved from the matrix when missing, at the cost of no large z-far support)? Should the user be allowed to provide either matrices and/or planes (the missing one being build from the other)?

• this PR actually enables unbounded z-far without z-fighting with the Forward+ renderer only (32bit SFLOAT reverse z-buffer). Compatibility and Mobile (used by many XR apps) only have 24bit UNORM reverse z-buffer that is more subject to z-fighting with large far distances. We discussed the opportunity to move to 32bit SFLOAT on Mobile too but that's a much broader topic.

• both Implement multi-layer projection camera #116424 and this PR will likely require @clayjohn's perspective on the orientation and the target architecture we choose for cameras and projections.

[Flarkk]

Rebased on top of #116752

[Flarkk]

Rebased

[3DartBlade]

Could I please get some guidance on how to use this in my projects, and/or if this is something I should expect in an upcoming engine update?
I'm unfortunately unfamiliar with terms such as 'PR' and the Godot engine's workflow, but I would be very interested in using this!
Thanks in advance!

[Calinou]

Could I please get some guidance on how to use this in my projects

Check Testing pull requests if you want to test this locally. Note that if you want to use this change in an exported project, you'll also need to compile export templates for all platforms you plan to export to.

and/or if this is something I should expect in an upcoming engine update?

4.7 is now in feature freeze, so any new features can only be merged in 4.8 at the earliest.