skip to main content
Idioma:
Primo Search
Clear Search Box
Search in: Busca Geral
Busca Avançada
Busca por Índices

Variable Binding for Sparse Distributed Representations: Theory and Applications

Frady, Edward Paxon ; Kleyko, Denis ; Sommer, Friedrich T.

IEEE transaction on neural networks and learning systems, 2023-05, Vol.34 (5), p.2191-2204

United States: IEEE

Texto completo disponível

Citações Citado por
  • Título:
    Variable Binding for Sparse Distributed Representations: Theory and Applications
  • Autor: Frady, Edward Paxon ; Kleyko, Denis ; Sommer, Friedrich T.
  • Assuntos: Brain ; Classification ; Cognition ; cognitive reasoning ; Compounds ; compressed sensing (CS) ; Convolution ; Data structures ; Neural Networks, Computer ; Neurons ; Neurons - physiology ; Problem Solving ; sparse block-codes ; sparse distributed representations ; Sparse matrices ; tensor product variable binding ; Tensors ; vector symbolic architectures (VSAs) ; Vectors
  • É parte de: IEEE transaction on neural networks and learning systems, 2023-05, Vol.34 (5), p.2191-2204
  • Notas: ObjectType-Article-1
    SourceType-Scholarly Journals-1
    ObjectType-Feature-2
    content type line 23
  • Descrição: Variable binding is a cornerstone of symbolic reasoning and cognition. But how binding can be implemented in connectionist models has puzzled neuroscientists, cognitive psychologists, and neural network researchers for many decades. One type of connectionist model that naturally includes a binding operation is vector symbolic architectures (VSAs). In contrast to other proposals for variable binding, the binding operation in VSAs is dimensionality-preserving, which enables representing complex hierarchical data structures, such as trees, while avoiding a combinatoric expansion of dimensionality. Classical VSAs encode symbols by dense randomized vectors, in which information is distributed throughout the entire neuron population. By contrast, in the brain, features are encoded more locally, by the activity of single neurons or small groups of neurons, often forming sparse vectors of neural activation. Following Laiho et al. (2015), we explore symbolic reasoning with a special case of sparse distributed representations. Using techniques from compressed sensing, we first show that variable binding in classical VSAs is mathematically equivalent to tensor product binding between sparse feature vectors, another well-known binding operation which increases dimensionality. This theoretical result motivates us to study two dimensionality-preserving binding methods that include a reduction of the tensor matrix into a single sparse vector. One binding method for general sparse vectors uses random projections, the other, block-local circular convolution, is defined for sparse vectors with block structure, sparse block-codes. Our experiments reveal that block-local circular convolution binding has ideal properties, whereas random projection based binding also works, but is lossy. We demonstrate in example applications that a VSA with block-local circular convolution and sparse block-codes reaches similar performance as classical VSAs. Finally, we discuss our results in the context of neuroscience and neural networks.
  • Editor: United States: IEEE
  • Idioma: Inglês
Links
Voltar para lista de resultados

  • Realização: ABCD-USP USP   Logos de Redes Sociais: Freepik

Buscando em bases de dados remotas. Favor aguardar.