High-Dimensional Memory Models: Transforming Real World Information into Semantics

Curt Burgess

University of California, Riverside

Global co-occurrence memory models, such as the Hyperspace Analogue to Language (or HAL), encode the symbols (words) in language input. In HAL, learning involves the encoding of weighted word co-occurrences in a moving n-width window into a memory matrix. A broader range of co-occurrence information is captured with this methodology than with local co-occurrence approaches. These weighted co-occurrence patterns form the basis of high-dimensional word meaning vectors which have the characteristics of distributed representations: graceful degradation, concepts formed by a large array of elements, and straightforward generalization. An advantage of these models is that they are operationally transparent and use learning procedures that scale up to real world language problems. The HAL model has been used to investigate a wide range of cognitive phenomena (associative and semantic priming, semantic and grammatical categorization, connotative definitions, semantic judgments, parsing constraints, deep dyslexia, cerebral asymmetries, concept acquisition, aspects of aging and development, and decision making) -- some of these results will be presented. Part of this presentation will entail a discussion of a range of memory metrics that have proven useful in theorizing about memory function and may be candidates for a more complete model of memory. High-dimensional memory models have been unjustly criticized recently by some in the embodied cognition camp. This is usually accomplished by presenting data that shows some kind of stimulus-response interference effect (these results have been around for decades) and then claiming that high-dimensional memory models are vacuous in principle. The more sophisticated thinker will realize that high-dimensional memory models are in fact a real first-approximation to a model with embodied cognition. It will be argued that this is a better starting point than positing elaborate and opaque models with unspecified representations and processes (see http://hal.ucr.edu/reprintPDFs/BurgGRrejoiner2000.pdf).