On inductive biases in vision transformers

Inductive biases have long been regarded as fundamental to the design of effective computer vision models, a perspective reinforced by the demonstrated success of convolutional neural networks. Guided by this view, the initial stages of this thesis concentrate on embedding explicit biases into transformer architectures to secure strong empirical performance. As the research advances, however, this assumption is systematically reassessed. Evidence accumulated across multiple studies indicates that, although inductive biases can facilitate convergence and improve sample efficiency, they are not universally beneficial and may, in certain contexts, restrict representational capacity. On the basis of these findings, this thesis contends that inductive biases are best understood as context-dependent design instruments which are indispensable under constraints of data or computation, yet less critical in large-scale regimes where model flexibility enables the direct emergence of structure from data.