Paper Explained - GLOM: How to represent part-whole hierarchies in a neural network (by Geoff Hinton, Full Video Analysis)

Geoffrey Hinton describes GLOM, a Computer Vision model that combines transformers, neural fields, contrastive learning, capsule networks, denoising autoencoders and RNNs. GLOM decomposes an image into a parse tree of objects and their parts. However, unlike previous systems, the parse tree is constructed dynamically and differently for each input, without changing the underlying neural network. This is done by a multi-step consensus algorithm that runs over different levels of abstraction at each location of an image simultaneously. GLOM is just an idea for now but suggests a radically new approach to AI visual scene understanding.

0:00 - Intro & Overview
3:10 - Object Recognition as Parse Trees
5:40 - Capsule Networks
8:00 - GLOM Architecture Overview
13:10 - Top-Down and Bottom-Up communication
18:30 - Emergence of Islands
22:00 - Cross-Column Attention Mechanism
27:10 - My Improvements for the Attention Mechanism
35:25 - Some Design Decisions
43:25 - Training GLOM as a Denoising Autoencoder & Contrastive Learning
52:20 - Coordinate Transformations & Representing Uncertainty
57:05 - How GLOM handles Video
1:01:10 - Conclusion & Comments

Paper: [2102.12627] How to represent part-whole hierarchies in a neural network