Learning causal representations in spatio-temporal systems

This thesis explores learning causal structures, mechanisms, and representations from high-dimensional, unstructured data using machine learning, focusing on visual and temporal systems. First, the thesis introduces ENCO, a novel algorithm for neural causal discovery with interventional data. ENCO reformulates graph search as an optimization of independent edge likelihoods, guaranteeing convergence and efficiently scaling to large graphs while handling deterministic variables and latent confounders. Next, the work tackles causal representation learning, developing CITRIS, a neural network-based method that identifies both scalar and multidimensional causal factors from high-dimensional time series data with interventions. This includes establishing a connection between causal representation learning and intervention design, determining the minimal interventions needed for identification. The method is further extended to handle instantaneous effects with iCITRIS, which simultaneously identifies causal variables and learns their instantaneous causal graph. The thesis then explores agent-based frameworks, introducing BISCUIT, a variational autoencoder that unsupervisedly learns both causal variables and agent interactions in complex environments, demonstrating applicability to embodied AI and robotics. Finally, the work investigates dynamical systems described by partial differential equations (PDEs). Analyzing temporal rollout strategies for neural PDE solvers, it identifies inaccurate modeling of non-dominant spatial frequencies as a key issue, causally impacting long-term dynamics. To address this, PDE-Refiner, a diffusion-inspired approach, is introduced to refine the modeling of all frequency components.