ABSTRACT
This chapter emphasises the significance of context in language and urban environments, drawing parallels between urban networks and neural networks. By examining the key operations in large language models, I demonstrate how they capture context through semantic embeddings transformed by data in a “context window,” analogous to short-term memory. Using an urban neighbourhood, I map physical features as tokens and employ a neural network to simulate context as spatial proximities, generating plausible city paths analogous to sentence construction. Adapting this exercise to word sequences, I develop a neural network to generate sentences, revealing parallels between urban contexts and language models, highlighting context’s role in both fields.
