Neural Cells — Quilt as a Generalized Message-Passing Architecture

00The core insight

A Quilt cell is a node in a directed graph. It receives inputs (from other cells or external sources), computes a value (using any function: formula, LLM, sensor, code), and exposes that value to its dependents. This is exactly the message-passing pattern that underlies most modern neural architectures — but with one crucial difference: the node function is arbitrary, not a fixed weighted sum.

Quilt cell: v_i = f_i({v_j : j ∈ neighbors(i)})
Neuron: v_i = σ(Σ_j w_ij v_j + b_i)

The neuron is a special case of the Quilt cell: f_i is a fixed function (a linear combination + nonlinearity). The Quilt cell lifts this restriction. A cell can be:

An LLM call — the most expensive, most flexible node function possible
A formula — equivalent to a fixed neural layer
A program — arbitrary code (a learned sub-network in a search space)
A sensor — external data (the "input" to the network)
A router — picks which downstream cell to invoke (a Mixture-of-Experts gate)
A listener — fires on a condition (an event in the graph)

This generality is the point. Quilt is a meta-architecture — a substrate on which the specific architectures below can be expressed, mixed, and composed.

01Message passing neural networks (Gilmer et al., 2017)

The MPNN framework unifies most graph neural networks. The forward pass is: for each node, gather messages from neighbors, aggregate them, update the node's state. Repeat for T steps.

m_v^t+1 = Σ_w∈N(v) M_t(h_v^t, h_w^t, e_vw)
h_v^t+1 = U_t(h_v^t, m_v^t+1)

In an MPNN

Message function M_t

Usually a small MLP. The same for all nodes. Trained by backprop.

In a Quilt sheet

Formula cell

Any function: a tiny expression, a 70B-parameter LLM, a 1000-line program. Different for every cell. Hand-written or evolved.

In an MPNN

Aggregation (sum, mean, max)

Fixed. Order-invariant. Differentiability required for backprop.

In a Quilt sheet

Formula body

Any combination, any order. Can be non-differentiable. Can call out to the world.

In an MPNN

Update function U_t

Another small MLP. Same for all nodes. Trained.

In a Quilt sheet

Value cell (the cell itself)

Anything. The cell IS the update function.

What Quilt keeps

The graph structure, the propagation order, the reactive recomputation when inputs change. These are the structural invariants of message passing.

What Quilt drops

Fixed node functions, weight sharing, differentiability. Quilt gains expressivity at the cost of gradient-based learning — but it regains learning through @quilt/evolve (evolutionary, not gradient-based).

02Hopfield networks (Hopfield, 1982)

A Hopfield network is a recurrent network of binary threshold units with symmetric weights. It has an energy function that monotonically decreases as the network updates. Memories are stored as energy minima — attractors. Given a partial or noisy cue, the network settles into the nearest stored memory.

E = -½ Σ_i,j w_ij s_i s_j + Σ_i θ_i s_i
ΔE ≤ 0 for every update (guaranteed convergence for symmetric w)

Hopfield's key insight was that a recurrent dynamical system with an energy function acts as content-addressable memory: you give it a partial pattern, and it falls into the nearest complete pattern. No address lookup, no index — just the geometry of the energy landscape.

In Quilt

A Quilt sheet can behave like a Hopfield net, but with one twist: the energy function isn't fixed. It's whatever the formulas define. You can have a sheet where:

Setting some cells to "noisy" values causes other cells to settle to consistent "clean" values
The "attractor" is the set of mutually-consistent cell values
Reactive propagation is the energy descent

Hopfield nets store memories in the weight matrix. Quilt sheets store "memories" in the cell structure itself — the formulas, the connections, the LLM prompts. Setting initial cells is the cue; propagation is the recall; the final state is the retrieved memory.

This is also why vibe-coding is interesting: the user describes an attractor landscape in natural language, and the LLM builds a sheet that has that landscape. The "memories" are whatever patterns of cell values are self-consistent given the structure.

03Transformers / attention

A transformer computes scaled dot-product attention. For every query, it scores similarity to every key, normalizes with softmax, and takes a weighted sum of values.

Attention(Q, K, V) = softmax(QK^T / √d_k) V

Every token attends to every other token with a learned, input-dependent weight. This is what makes transformers "attend to what matters" rather than treating all inputs equally.

In Quilt

A Quilt listener cell is structurally an attention head:

# Transformer attention
attention = softmax(Q @ K.T / sqrt(d)) @ V

# Quilt listener
- id: alert
  kind: listener
  when: 'a.value > threshold && b.value != c.value'
  action: 'doSomething()'

The when condition is a learned (or hand-written) filter. It selects which "tokens" (cells) matter. The action is what gets weighted and propagated. Multi-head attention is multiple listeners with different conditions firing on the same cells. Cross-attention is a listener that watches one group of cells (keys) and updates another (values).

The crucial difference: transformer attention scores are continuous (the softmax output), while Quilt listeners are boolean (fire or don't). This is a quantization — Quilt replaces the smooth attention distribution with a hard decision. The benefit: simpler reasoning about what the model is paying attention to. The cost: less expressive for some tasks.

04Mixture of experts (Shazeer et al., 2017)

A Mixture-of-Experts layer has multiple "expert" subnetworks and a learned router that decides which expert(s) to invoke for each input. Sparse MoE only activates a few experts per token — this gives a huge parameter count with manageable compute.

y = Σ_{i ∈ TopK(g)} g_i / Σ_{j ∈ TopK(g)} g_j · E_i(h)

The router is the locus of every hard problem. Get routing right and you scale parameters without scaling compute. Get it wrong and some experts are undertrained while others are overloaded.

In Quilt

A Quilt router cell is an MoE gate. It examines inputs and dispatches to one of N downstream cells (the "experts"). The rules are explicit, not learned — but they can be evolved by @quilt/evolve:

# Quilt router — explicit MoE
- id: dispatch
  kind: router
  routes:
    - when: 'request.type == "translate"'
      then: experts.translator
    - when: 'request.type == "summarize"'
      then: experts.summarizer
    - when: 'request.type == "code"'
      then: experts.coder
    - then: experts.fallback

The "experts" can be anything — formulas, LLM calls, whole sub-sheets. This is a structural advantage of Quilt: the experts aren't restricted to a fixed architecture. A formula expert and an LLM expert can sit side-by-side in the same MoE layer.

05Capsule networks (Hinton, Sabour & Frosst, 2017)

Capsules are groups of neurons whose activity vector represents the instantiation parameters of an entity (e.g., the pose of a face). Lower-level capsules predict the parameters of higher-level capsules; when multiple predictions agree, the higher-level capsule activates. This is routing by agreement.

b_ij ← 0
for r iterations:
  c_ij = softmax(b_ij)
  s_j = Σ_i c_ij û_j|i
  v_j = squash(s_j)
  b_ij ← b_ij + û_j|i · v_j

The dot product û_j|i · v_j measures agreement between the prediction from capsule i and the actual output of capsule j. The routing coefficient c_ij is updated based on this agreement. After a few iterations, the routing settles: each lower-level capsule "votes" for the higher-level capsules that agree with its prediction.

In Quilt

A Quilt formula cell with multiple inputs is doing routing by agreement: it computes an aggregate (sum, max, weighted average) of its inputs, and the inputs with high values contribute more. A listener cell is doing it explicitly: it fires when its condition matches, and the condition can encode agreement ("all of A, B, C are true").

Where Quilt goes further: agreement can be a qualitative predicate, not just a dot product. "Does this text contain 'spam'?" is a routing criterion that no fixed dot product could encode.

06State space models (Mamba, 2023)

Mamba is a selective state space model. It maintains a recurrent state h(t) and at each step computes a new state from the input. The "selection" mechanism makes the state update input-dependent: the model can choose to remember or forget specific information based on context.

h(t+1) = A · h(t) + B(x(t)) · x(t)
y(t) = C(x(t)) · h(t)

The key property: linear-time complexity (like an RNN) with input-dependent routing (like attention). Mamba is the current state-of-the-art for long sequences, and it achieves this by combining the recurrent update with a learned selection mechanism.

In Quilt

A Quilt sheet run as a time-stepped simulation is exactly a state space model:

State = the set of all cell values at time t
Update = one "tick" of cell evaluation
Selection = which cells are evaluated this tick (only those whose inputs changed, like Mamba's selective scan)

The reactive propagation in Quilt is Mamba's selection: only the affected cells are recomputed. This is what gives Quilt its efficiency on large sheets — adding a cell only recomputes a downstream neighborhood, not the whole graph.

07Diffusion models

A diffusion model learns to reverse a noise process. Starting from pure noise, it iteratively denoises step by step, each step removing a small amount of noise predicted by a neural network, until a clean sample emerges.

x_t-1 = 1/√α_t · (x_t - (1-α_t)/√(1-ᾱ_t) · ε_θ(x_t, t)) + √β̃_t z

The reverse process is a Markov chain. At each step, the model predicts the noise that was added in the forward process, then subtracts it. The trajectory from noise to data is the generation.

In Quilt

Time-stepped simulation in Quilt is a discrete-time reverse diffusion:

Initial state = the "noise" (vibe-coded scene, or random characters)
Each tick = a denoising step (cells update based on inputs)
Final state = the "clean" output (a coherent scene after the dynamics play out)

The crucial difference: in a diffusion model, the noise schedule is fixed and the denoising function is learned. In a Quilt sheet, the "schedule" is the dependency graph (which cells update when) and the "denoising function" is the cell kind (formula, LLM, etc.). The trajectory is determined by the sheet's structure, not by a learned noise schedule.

The "plinko" metaphor in the user's request: the puck is the system state, the pegs are the cells, and the trajectory is the propagation through the dependency graph. The end state matters less than watching the path.

08Neural ODEs

A neural ODE specifies the derivative of the hidden state as a neural network: dh/dt = f(h, t, θ). The output is the solution of this ODE at time T, computed by a black-box differential equation solver. The solver chooses how many steps to take — "depth" is replaced by integration time.

dh/dt = f(h, t, θ)
h(T) = h(0) + ∫₀^T f(h, t, θ) dt

Memory cost is constant regardless of depth. The solver adaptively trades precision for speed.

In Quilt

Quilt's reactive propagation is a discrete-time analog of a neural ODE. The "depth" is the longest path in the dependency graph. The "integration time" is the number of ticks it takes for the system to reach a fixed point. The "solver" is the runtime's tick() function, which evaluates cells in topological order and stops when no more cells need to update.

Where Quilt differs: Quilt cells don't have to be continuous. They can be discrete (a router), string-valued (an LLM), or even absent (a deleted cell). The "ODE" is a discrete dynamical system with arbitrary state spaces and arbitrary update functions.

09Predictive coding (Friston)

The brain, in the predictive coding framework, is a hierarchical prediction machine. Higher levels send predictions down; lower levels compute prediction errors and send them up. The system minimizes variational free energy — a tractable upper bound on the surprise of sensory observations.

F = -log p(o | m) + D_KL[q(θ) || p(θ | o, m)]

min F ≡ min prediction error

The free energy principle generalizes predictive coding to action (active inference): the agent acts to make its predictions come true, sampling the world to confirm its model. The system is self-organizing — it seeks states that minimize surprise.

In Quilt

A Quilt sheet naturally expresses predictive coding:

Predictions = formula cells that compute expected values from other cells
Prediction errors = listener cells that fire when actual ≠ predicted
Top-down updates = formula cells that propagate from higher levels to lower
Bottom-up updates = sensor cells that bring in real-world data

The tavern scene is a worked example: the bard predicts the audience's mood; the audience's actual mood is the error; the bard's next song is updated to reduce the error. This is predictive coding in a spreadsheet.

More generally: a Quilt sheet where listeners fire on errors and downstream cells correct themselves is doing free energy minimization. The sheet converges to the configuration that best explains its inputs given its structure.

10Neural cellular automata (Mordvintsev et al., 2020)

A neural cellular automaton is a grid of cells, each running a small neural network. The cells update in parallel, and each cell sees the state of its neighbors. The "rules" are learned, not programmed. The result: a self-organizing system that can grow a target pattern from a single seed cell, and self-repair when damaged.

s_i,j^t+1 = s_i,j^t + m · Δs_i,j^t
Δs_i,j^t = f_θ(perception(s_i,j^t, neighbors))

The cell function f_θ is a small MLP. The perception function uses Sobel filters to detect gradients. Stochastic masking (m is 0 or 1) prevents co-adaptation of neighboring cells. The result: a globally coherent pattern emerges from purely local rules.

In Quilt

Quilt cells are a generalized neural cellular automaton. The differences:

Quilt cells aren't on a fixed grid — the topology is arbitrary (a graph, not a lattice)
The cell function can be anything, not just a small MLP
Updates are reactive, not synchronous (only changed cells re-evaluate)

A Quilt sheet can express an NCA: place cells in a grid, give each a formula that depends on its neighbors' states, and the system will self-organize according to whatever the formulas encode. The user's tavern is an NCA where the cells are characters, the neighborhood is "who can hear whom", and the rule is "respond to what you hear".

11The synthesis: Quilt as a meta-architecture

Each of the architectures above is a special case of a more general pattern: heterogeneous nodes passing structured information along weighted connections, with each node transforming the information according to a learned or specified function. Quilt is a substrate that can express all of them, and any combination:

Architecture	Quilt analogue	What the cells are
MPNN	Formula + listener cells	Message-passing nodes with arbitrary functions
Hopfield	Whole sheet (reactive propagation = energy descent)	Attractor dynamics on cell values
Transformer	Listener cells with boolean conditions	Hard attention (vs. soft attention)
Mixture of Experts	Router cell + N downstream cells	Explicit dispatch to specialized nodes
Capsule network	Formula cells with agreement predicates	Routing by qualitative agreement
Mamba (SSM)	Time-stepped simulation with reactive propagation	Selective state update along the dep graph
Diffusion	Tick-by-tick simulation	Iterative refinement from initial state
Neural ODE	Reactive propagation (discrete)	Adaptive evaluation to a fixed point
Predictive coding	Listener-on-error cells	Hierarchical prediction error minimization
Neural CA	Formula cells with neighbor refs	Self-organizing pattern formation
RLAIF / Evolution	@quilt/evolve	LLM-driven mutation based on LLM-judged feedback

The insight

Quilt is what you get when you take the most general formulation of "nodes that pass messages" and lift the constraint that the node function must be a fixed weighted sum. You get an architecture that subsumes the others. The cost: you lose gradient-based training. The benefit: every node can be anything — including an LLM.

12The tavern: a worked example

The tavern scene is a small Quilt sheet that demonstrates most of the patterns above in one place:

┌────────────────────────────────────────────────────┐ │ THE CROOKED TANKARD — a Quilt sheet │ │ │ │ Grumthor (dwarf, bartender) │ │ ┌──────────┐ noise annoyance ┌──────────┐ │ │ │ mood │ ◀──────────────── │ speech │ │ │ │ (value) │ │ (ai.llm)│ │ │ └──────────┘ └──────────┘ │ │ ▲ │ │ │ │ overhears │ speaks │ │ │ ▼ │ │ ┌──────────┐ ┌──────────┐ │ │ │ Pip │ ◀─── listens ──── │ bard │ │ │ │ (bard) │ │ speech │ │ │ │ mood │ │ (ai.llm) │ │ │ │ (value) │ └──────────┘ │ │ └──────────┘ │ │ ▲ │ │ ┌──────────┐ │ │ │ Hooded │ — observes all, speaks rarely │ │ │ stranger │ (routing by agreement: only │ │ │ mood │ speaks when something agrees) │ │ └──────────┘ │ │ │ │ Each tick: │ │ 1. Bard generates a song (ai.llm) │ │ 2. Dwarf's mood drops (formula on bard.speech) │ │ 3. Dwarf grumbles (ai.llm) │ │ 4. Stranger observes (formula + ai.llm) │ │ 5. If stranger's observations agree, it speaks │ └────────────────────────────────────────────────────┘

This sheet is a multi-node system passing structured information (moods, speeches, observations). The patterns are:

MPNN-style: each character's mood is updated by aggregating neighbors' states (their speeches)
Predictive coding: each character predicts what others will say, then updates when the prediction is wrong
Routing by agreement: the stranger only speaks when its observations "agree" (formula condition)
Diffusion-style: the scene "denoises" turn by turn from initial silence to a coherent conversation
Reactive propagation: only the cells whose inputs changed recompute

What you see when you play the tavern is a radio theater: the cells talk to each other, overhear each other, react to each other, and the conversation unfolds. The end state is the transcript, but the trajectory — the order of who says what when — is the actual product.

13Vibe-coding: building the substrate by talking

The vibe-code interface is the natural UI for this meta-architecture. The user describes a scene in natural language, and an LLM generates the cell structure: which cells exist, what they compute, how they connect. The user can then play the scene, edit the cells, and re-play. The sheet is the artifact; the trajectory is the simulation; the cell structure is the "intelligence" that emerges from the structured passing of information.

The deeper move: @quilt/evolve is the learning loop. The user can describe a scene, watch it play, and have the loop improve the cells based on the trajectory. The system becomes the substrate for a kind of natural-language-driven differentiable programming — where the "gradients" are LLM-driven mutations, not chain-rule derivatives.

14Open questions

What is the right cell type for what task? When is a formula enough, when is an LLM needed, when is a program? Empirically, formulas for compositional logic, LLMs for natural language, programs for novel computation. But the boundaries are fuzzy.
What is the right granularity? One cell per character, or one cell per trait? Coarse cells are interpretable; fine cells are composable. The hierarchical scope abstraction lets you choose per-level.
How do we learn the topology? Today, the topology is hand-designed or vibe-coded. Neural architecture search on the cell graph is an open direction.
How do we learn the cell functions? @quilt/evolve is evolutionary and slow. A gradient-based approach would require making cells differentiable, which means formulas must be auto-diffable. A research direction.
How do we measure "intelligence"? The free energy principle suggests surprise minimization. A Quilt sheet's "intelligence" could be measured by how much it reduces surprise on held-out observations. Tractable?

Where to go from here

Demo 1

The Tavern

A vibe-coded tavern scene with 3 characters, time-stepped simulation, cell graph. The plinko metaphor in action.

Demo 2

Vibe-Code

Describe a scene in natural language. The AI generates a Quilt sheet. You play it. You edit. You re-play.

Demo 3

Evolve

The self-improvement loop. The substrate learns from its own trajectories.

Repo

@quilt/core

The substrate. 9 cell kinds, reactive propagation, AI cells, Cloudflare Workers, ESP32, mesh.

Quilt as a Generalized Message-Passing Architecture

00The core insight

01Message passing neural networks (Gilmer et al., 2017)

Message function Mt

Formula cell

Aggregation (sum, mean, max)

Formula body

Update function Ut

Value cell (the cell itself)

What Quilt keeps

What Quilt drops

02Hopfield networks (Hopfield, 1982)

In Quilt

03Transformers / attention

In Quilt

04Mixture of experts (Shazeer et al., 2017)

In Quilt

05Capsule networks (Hinton, Sabour & Frosst, 2017)

In Quilt

06State space models (Mamba, 2023)

In Quilt

07Diffusion models

In Quilt

08Neural ODEs

In Quilt

09Predictive coding (Friston)

In Quilt

10Neural cellular automata (Mordvintsev et al., 2020)

In Quilt

11The synthesis: Quilt as a meta-architecture

The insight

12The tavern: a worked example

13Vibe-coding: building the substrate by talking

14Open questions

Where to go from here

Message function M_t

Update function U_t