Relighting And Color Grading With Machine Learning

Continuity can be a challenge for shoots that are plagued by varying weather conditions, where, for instance, the pick-up shots are in bright sunlight but the core footage was shot under an even layer of cloud.

The cheapest and perhaps most notorious post-fix for 'night-time' footage, familiar to any viewer of old movies, was altering the F-stop settings at processing time in order to over-expose the film and produce a dark and gloomy effect from material that was shot in broad daylight – known as 'day for night'[1].

Neural networks have been brought to bear on the problem for a few years now. In 2019 a Google-led academic collaboration presented a novel neural network that implemented a rudimentary process for relighting, though the results were not entirely convincing[2].

In 2020 another collaboration, this time between Amazon, Adobe and the University of Maryland, developed a relighting algorithm capable of working on portraits as large as 1024x1024 – which is pretty HD for the image synthesis space, at least at the moment.