Photometric invariance for intrinsic image decomposition

Intrinsic image decomposition is the process of recovering the image formation components (reflectance and shading) from an image. Previous methods employ either explicit priors to constrain the problem or implicit constraints as formulated by their losses (deep learning). As shading transitions are generally smoother than reflectance (albedo) changes, these methods may fail in distinguishing strong photometric effects from reflectance variations. These can also be negatively influenced by strong illumination conditions causing shading-reflectance leakages. Recently, deep learning methods have also been explored, powered by large scale datasets like NED, MPI Sintel, GTA V, IIW, MIT Intrinsic Images, 3DRMS and SRD. However, these methods are dependent on the representation capabilities of the datasets and do not learn the physics of the image formation model.
In this thesis, photometrically invariant properties are studied for the purpose of IID. First the shading is constrained in the form of extending the image formation model for a finer image formation to distinguish strong photometric effects from reflectance variations. Following this, photometric invariant priors along with statistical priors are also explored for constraining the reflectance. Finally, the invariants are integrated into a global and local learning through the use of transformers.
Large scale experiments show that the proposed methods result in an efficient and physically grounded learning of the IID problem. Additionally, the networks are trained on synthetic data and tested on real world images showing generalisation capabilities due to the use of photometric invariants. Finally, a new fully ray traced realistic indoor dataset is also proposed.