Learning structured representations of objects and relations

This thesis explores how we can enable more human-like intelligence in machine learning systems by helping them overcome the binding problem. More specifically, we want to enable neural networks to flexibly and dynamically represent and relate entities. The thesis is divided into two parts. In the first part, we explore inductive biases in graph-based representations. In the second part, we develop and investigate a novel representational format to tackle the binding problem.
Our contributions are as follows:
1) We propose Amortized Causal Discovery (ACD) - a novel framework for causal discovery in which we learn to infer causal relations across samples with different underlying causal graphs but shared dynamics.
2) We propose the Complex AutoEncoder (CAE) - an object discovery model that utilizes a novel representational format for objects. By incorporating complex-valued activations into a convolutional autoencoder, the CAE learns to represent object properties in the activations' magnitudes and object affiliations in their phase values.
3) We propose Rotating Features - an extension of the Complex AutoEncoder which scales this approach from simple toy to real-world data.
4) We propose a novel cosine binding mechanism for Rotating Features. This allows us to better understand the dynamics necessary for Rotating Features to learn to separate objects using their orientation values.