Rendering Real Time Depth of Field Effects

Abstract

In 2020, Lynn Asberg defended her master's thesis on replicating realistic Depth Of Field effects using Seidel Aberrations as post-process to an already rendered image with a depth map. In her thesis, she presented a novel approach to efficiently render both monochromatic and chromatic aberrations starting from real lens specifications to obtain authentic-looking bokeh. To do this, and to give the end user some creative control, she reduced the lens to 7 parameters that can be changed at runtime to alter the obtained effect.

Asberg implemented her algorithm as a post-process effect, mentioning that reversing the procedure to generate primary rays for a ray tracer would work just as well, making it usable as a real-time Depth Of Field simulation.

This brings us to our first research question: RQ1: Can we reverse Asberg's innovative approach and expand it in order to produce primary rays for breda, Traverse Research's real-time rendering framework?

This research question was ultimately answered negatively. This reversal faced a lot of issues due to the intricate aberrations experienced by the individual rays within the optical system and the complex nature of light interactions. Those roadblocks made the reversibility of Asberg's technique non-trivial and prevented its direct application to the real-time rendering framework.

To still deliver a Depth Of Field effect, while also attempting to solve the partial visibility issue that affect any post-process effect, we turned to explore different techniques and methods to render practical DoF effect. The paper from Tan et al. introduced an interesting hybrid approach in which the missing geometry gets ray traced after a post-process effect inspired by Jimenez. This leads us to the second research question:

RQ2: Can we implement Tan et al. paper in breda so that a post-process Depth Of Field can be rendered in real time, with low latency and while also improving on existing techniques for the partial visibility issue?

Therefore, we moved to implement Tan et al. paper. The complete lack of access to the code, the fact that the authors were not available to answer any of the problems that came up during development and because some sections of the paper left a lot of guess work to do this implementation was not trivial. Those problems made it so that it was not possible to reach the same result as shown in Tan et al.. Nonetheless, part of the work was merged into the main branch of breda, and the final implementation still shows how the approach can actually improve on existing research.