
Neuro-symbolic learning seeks to combine the ability of deep neural network to process the vast amounts of features extracted from the real-world with symbolic representations to encapsulate and identify important patterns. Often this learning is in virtual worlds, such as theĀ CLEVR datasetĀ (see example picture left), which can capture the relationship between objects but not the physical instantiation of them. It ignores the physical sensory-motor (sensorimotor) learning that is crucial to embodied agents to interact with and learn about the world. This PhD will study how neural-symbolic learning can be advanced through sensorimotor learning.