Hopfield Networks
A Hopfield network is a content-addressable memory: store a handful of binary patterns with a one-shot Hebbian update, then hand the network a corrupted version and watch it roll downhill, strictly downhill, into the nearest stored pattern. Below you can store patterns, paint your own queries, watch the energy descend, and explore the basins of attraction in projected state space.
Store patterns with a Hebbian outer product
Toggle prototypes in and out of memory. The 100×100 grid on the right is the symmetric weight matrix , where green is excitatory and red is inhibitory. Past patterns the network starts confusing its own memories.
Prototype library
Weight matrix W (100×100)
Corrupt a pattern, watch it heal
Pick a target prototype, set the corruption level, then run. Each step picks a neuron and snaps it to the sign of its local field . The energy is a Lyapunov function: under asynchronous updates it never increases. You can also paint directly on the query grid by clicking cells to flip them.
The energy landscape, projected to 2D
The state lives in , that is, corners of a hypercube. To visualize basins of attraction we pin each stored prototype to a vertex on the unit circle and project every state to the weighted sum of vertices, weighted by its overlap with each prototype. Random initial states cluster near the origin and flow radially outward into the nearest stored vertex. Spurious attractors (mixtures the network learned by accident) appear as extra orange dots between the prototype vertices. Click anywhere to drop the network at that point and watch its trajectory.
Drag to rotate, scroll to zoom. Click on the surface to drop the network. A small x marks your click, a dashed line shows the snap to the nearest binary corner (the network state lives on {±1}100, not on the continuous plane), and the solid tube is the dynamics descending into the deepest accessible well.
The 0.138 wall
Hopfield networks store roughly random patterns before recall collapses. Below the slider we store K random ±1 patterns, corrupt each with 10% noise, run async to convergence, and measure overlap with the original. Recall accuracy is the fraction with overlap ≥ 0.95.
The 1982 Hopfield network was retired by the field for two decades, since it stores too few patterns and gets trapped in spurious minima. In 2020, Ramsauer et al. resurrected the idea: replace the polynomial interaction with an exponential one and capacity becomes exponential in N. The update rule is identical, line for line, to the attention mechanism in a transformer. Every attention head is doing one step of modern Hopfield retrieval.
John Hopfield shared the 2024 Nobel Prize in Physics for this work.
- Hopfield, J. J. Neural networks and physical systems with emergent collective computational abilities. PNAS 79(8), 1982. doi:10.1073/pnas.79.8.2554
- Amit, D., Gutfreund, H., Sompolinsky, H. Storing infinite numbers of patterns in a spin-glass model of neural networks. Phys. Rev. Lett. 55, 1530, 1985. (The ≈ 0.138 result.)
- Ramsauer, H. et al. Hopfield Networks is All You Need, ICLR 2021. arXiv:2008.02217
- The Royal Swedish Academy of Sciences. Scientific background to the Nobel Prize in Physics 2024. nobelprize.org/2024
Everything in this demo runs in your browser. Source is on GitHub.