Complex adaptive systems · arXiv 2026
Self-organization to the edge of ergodicity breaking in a complex adaptive system
National University of Singapore (Physics · NUS Cities · Risk Management Institute · Mathematics) · A*STAR Singapore
Give a population of learning agents a hard, rugged problem, start every agent hot, and let the least fit be replaced again and again. Nobody tunes anything, yet the population settles exactly where its learning dynamics is about to lose ergodicity, and there it solves the problem better than any hand-tuned setting.
The claim
Evolution parks learning agents at the edge of ergodicity breaking, because only there can well-adapted agents keep being replaced.
Optimal reward · the ergodicity edge · scale-free avalanches are three readings of one condition: the cascade a replacement sets off branches marginally, \(\sigma\simeq\rho(M)=1\).
Scroll the panel sideways →
Every agent starts hot, with an exploration temperature \(T\) between 1.4 and 2; each newcomer draws \(T\) uniformly from 0 to 2, and the memory is \(\alpha=0.75\). Left: the population, coloured by temperature, turns from red to mostly blue with a few warm agents. Top right: the collective reward climbs to the best level any single fixed temperature reaches (dashed line, read off Fig. 1 of our paper). Bottom right: temperatures pile up below the edge window and are depleted above it.
Two questions for the rest of the page: why does it stop there, and why is that state good?
0 · Context
Self-organized criticality, but for agents that learn
Self-organized criticality is well established for driven, dissipative systems with simple threshold rules: sandpiles, forest fires, the Bak–Sneppen model of evolution. Such systems tune themselves to a critical point, where events of every size occur, without anyone setting a parameter. Their units, however, are all alike, follow fixed rules and pursue nothing.
Homogeneous units, threshold rules, no objective.
Real adaptive systems, biological, social or economic, are made of heterogeneous agents that learn and pursue goals. That raises two questions:
- Do such systems self-organize to criticality without tuning?
- If so, which critical state do they reach, and why that one?
To answer them we built EvoSK, a minimal model made of three standard ingredients:
Sherrington–Kirkpatrick spin glass
A rugged landscape: random couplings \(J_{ij}\) between all pairs of agents, with exponentially many locally stable configurations.
Reinforcement learning
Each agent maximises its own local reward, exploring at its own temperature \(T_i\).
Bak–Sneppen replacement
At every step the least fit agent is removed and replaced by a newcomer with a random temperature.
The population reaches the edge of ergodicity breaking of its own learning dynamics. There it shows scale-free avalanches with exponent \(\tau\approx-3/2\) and a collective reward above that of every fixed-temperature setting. All three are one condition, a replacement cascade that branches marginally, and an exact bookkeeping rule of evolution forces the population onto it.
The argument in one picture
The top row is what we observe. The middle row is the one condition that explains all three observations. The bottom row is why evolution goes there. The rest of the page walks through it from top to bottom.
Selection has no choice: the only state it can sustain is the marginal one.
1 · Model
Learning on a rugged landscape, plus selection
The SK game
\(N\) agents are all connected to each other, with symmetric random couplings \(J_{ij}=J_{ji}\) drawn from a Gaussian of variance \(1/N\). Each agent \(i\) plays an action \(\sigma_i=\pm1\). It keeps one number, its incentive \(Q_i\), a moving average of its local field \(h_i\) (the summed pull of all other agents), with memory parameter \(\alpha\):
It then plays a softmax policy at its exploration temperature \(T\): a cold agent nearly always follows its incentive, a hot agent often does not. Its expected action is \(m_i\):
(Each agent really keeps a value for both actions; since both start at zero and receive opposite rewards, one number \(Q_i\) carries all the information.) Each agent's reward is its own local stability \(\lambda_i\): positive when it plays with its field, negative when it plays against it. Summed over agents, the collective reward is minus twice the SK energy \(E=-\tfrac12\sum_{ij}J_{ij}\sigma_i\sigma_j\):
So collective optimality is the classic hard problem of finding low-energy states of a spin glass. Because the couplings are symmetric, this is a potential game: when one agent switches action, its reward changes by exactly \(-\Delta E\). Learning therefore settles into satisficing states, mutual best responses in which every action is aligned with its field and no agent can gain by switching alone. There are exponentially many of them, almost all far above the ground state: individually optimal, collectively suboptimal.
Agents are coloured by their stability \(\lambda_i\): stable, marginal, misaligned. Chords show the strongest couplings. Low \(T\) freezes the population into one satisficing state; high \(T\) randomises it.
EvoSK: add Bak–Sneppen selection
Now temperature becomes a per-agent trait, and the population evolves. Every replacement step has three parts:
- Play. Every agent updates \(Q_i\) and acts at its own \(T_i\) (two updates per step).
- Evaluate. Fitness is \(f_i=\sigma_i Q_i\), the value of the action the agent took.
- Replace the least fit agent with a blank-slate newcomer: \(Q\to0\) and a fresh temperature drawn uniformly from \([0,2]\).
The kernel for new temperatures is flat, so any structure in the surviving temperatures is made by selection alone. The only knob left is the memory \(\alpha\); temperature is now the population's choice.
Every agent's reward is its own stability; the collective reward is the energy; temperature is left to evolution.
2 · Three signatures
Optimal, at the edge of ergodicity, and scale-free
1 · Evolution beats every tuned temperature
The long-run collective reward of EvoSK has a broad near-optimal plateau for memory \(\alpha\approx0.3\)–\(0.8\): long memory gets stuck, short memory over-explores. At \(\alpha=0.75\), EvoSK (purple band) lies above the SK game at every fixed temperature (black curve).
Freeze the evolved temperatures and rerun the plain SK game: it falls below the fixed-\(T\) optimum (green band). The gain is in the evolutionary dynamics, not in the traits it ends up with.
2 · The evolved population sits at the edge of ergodicity
Is the learning dynamics ergodic, meaning that independent runs forget where they started and agree on their long-run behaviour? We measure this with an ergodicity coefficient \(\epsilon\): the median total-variation distance between the steady-state reward distributions \(\mu,\nu\) of independent runs with the same temperatures,
\(\epsilon\approx0\) means ergodic (runs mix), \(\epsilon\approx1\) means ergodicity is broken (each run keeps its basin), and values in between mark the edge. To locate the evolved state, we rescale all evolved temperatures together, \(T_i\to T_i\,2^\lambda\), and sweep the perturbation strength \(\lambda\); \(\lambda=0\) is the evolved population itself.
For \(\alpha=0.5\)–\(0.8\) the evolved state sits where \(\epsilon\) changes fastest. At \(\alpha=0.3\) it lies on the breaking side, at \(0.9\) on the ergodic side.
The initial temperatures give \(\epsilon\approx0\) for every \(\alpha\): the starting population is ergodic. The edge is reached, not inherited.
3 · Scale-free avalanches, \(\tau\approx-3/2\)
As in the Bak–Sneppen model, we define an avalanche by the population's minimum fitness: it starts when the minimum drops by more than a threshold \(\delta\), and lasts until it has recovered. Its size \(S\) counts the agent updates spent below that level.
The size distribution \(P(S)\) is a power law over nearly four decades with exponent \(\tau=-1.58\) (quartiles \(-1.68\), \(-1.44\)), measured in the window of \(\delta\) where the estimate is stable, and similar for \(\alpha=0.5\)–\(0.8\).
A power law alone is not evidence of criticality. Our identification rests instead on how the system responds to a perturbation, measured and computed independently. That is the next section.
Three signatures. Coincidences, or one thing seen three ways?
3 · One condition
A replacement is a perturbation that branches
Sections 3 to 5 present our extended analysis of the mechanism, which goes beyond the first arXiv version of the paper.
Every replacement perturbs the population. The newcomer probably plays a different action from the agent it replaced, which shifts every local field by an amount of order \(1/\sqrt N\). Agents close to their threshold may flip in response, and every flip shifts the fields again. A replacement therefore triggers a branching process: its ratio \(\sigma\) is the mean number of further flips caused by one flip.
Scroll the panel sideways →
Coloured cells are where the perturbed copy differs from the original; faint cells have differed at some point. Cold (\(T=0.1\)): the damage heals. Hot (\(T=1.5\)): it becomes extensive. Near \(T=0.3\): most probes die, rare ones run far.
Heals
Each flip triggers fewer than one more.
Scale-free
Most probes die; rare ones run far.
Extensive
Surviving cascades reach a finite fraction of all \(N\) agents.
Shared randomness makes the expected damage linear
Softmax agents flip spontaneously all the time, so how can the damage be measured cleanly? We run two copies with identical couplings and identical random numbers, differing by one flipped action. Agent \(i\) then acts differently exactly when its shared uniform draw \(u_i\) falls between the two copies' policies, which happens with probability
The susceptibility \(\chi_i\) acts as a valve: it is open only for agents near their threshold. On average, damage then follows the mean update, in which each action is replaced by its expected value \(m_j\): a change \(\delta Q_j\) passes the valve \(\chi_j\), shifts the fields through \(J_{ij}\), and enters the incentives with weight \(\alpha\). The incentive differences evolve linearly,
where \(M\) is the Jacobian of the mean learning update.
The branching ratio is a spectral radius
The branching ratio is therefore set by the largest eigenvalue magnitude, the spectral radius, of the Jacobian:
When the population is hot, many valves are open and \(\rho(M)>1\): perturbations grow. When it is cold, the valves are shut and \(\rho(M)<1\): perturbations heal. Counting real flips instead of incentive damage gives a similar matrix, \(\operatorname{diag}(\chi)\,M\operatorname{diag}(\chi)^{-1}\), with the same spectrum. So a statement about cascades is also a statement about the stability of learning.
Scroll the panel sideways →
Eight 256-agent SK games at a uniform temperature are cooled step by step from \(T=1\) to \(0.05\). At each temperature the browser equilibrates them and computes the largest eigenvalue of \(M\) exactly, on several snapshots each. The grid shows each agent's valve \(\chi_i\), bright where open. Instantaneous values fluctuate strongly, so the averaging matters.
Approximation: \(M\) is treated as fixed over the steps of one cascade. Since \(\chi\) moves with \(Q\), \(\rho(M)\) only approximates the growth rate of the product \(M(t)M(t{-}1)\cdots\).
Marginal branching is the ergodicity edge
Both crossings sit inside the window where \(\epsilon\) falls from 0.85 to 0.15. The reason is direct: at a satisficing fixed point, \(M\) is the Jacobian.
\(\rho(M)<1\): every fixed point is an attractor → runs keep their basin → \(\epsilon>0\).
\(\rho(M)>1\): no fixed point is stable → runs mix → \(\epsilon\to0\).
A critical branching process has total progeny \(P(S)\sim S^{-3/2}\): the avalanche exponent is derived, not fitted.
At every memory level, three thresholds share one window
Is this special to \(\alpha=0.75\)? For each memory level, the grey bar is the independently measured \(\epsilon\) edge window. The purple dot is the balance boundary \(T_\times\), which comes from the evolution alone (next section); the gold star is \(\sigma=1\) from damage spreading; the green square is \(\rho(M)=1\) from the Jacobian. All three fall inside the window at every \(\alpha\), even though the windows differ a lot.
The ergodicity edge and scale-free avalanches are one condition: \(\sigma\simeq\rho(M)=1\).
4 · Why evolution goes there
Every arrival must die, cold agents included
Exact bookkeeping
Start from bookkeeping that holds exactly. One agent dies per step, and the newcomer's temperature comes from the flat kernel \(u(T)\). Writing \(\rho_t(T)\) for the temperature distribution of the population and \(\kappa_t(T)\) for the distribution of the temperatures of the agents that die,
In any stationary state the death flux must equal the injection, flat, however uneven the population is. Writing the death flux as \(N\rho\,h\) with a per-capita hazard \(h(T)\):
Temperatures pile up where the hazard is low. The natural boundary is where an agent's expected lifetime equals one turnover of the population, \(h(T_\times)=1/N\). Below \(T_\times\) agents outlive a turnover and are enriched; above it they are replaced within one and are depleted.
Scroll the panel sideways →
The standing density skews cold, the death flux stays flat, and the hazard rises by orders of magnitude, crossing \(Nh=1\) near \(T\approx0.3\)–\(0.4\). In our production runs (\(N=256\), \(3.25\times10^6\) events) the death flux is flat to 1.4% while \(\rho^*\) varies 440-fold, and \(T_\times\) lies inside the edge window at every \(\alpha\).
Cold agents must die too, and that takes two gates
Every arrival must die, including newcomers that become cold and well aligned; each of them must be removed at a rate of order \(1/N\). A cold agent can die only in two steps:
Strand
\(S\) persistent flips shift a bystander's field by a Gaussian amount with \(\mathrm{Var}(\Delta h)=4S/N\). Reversing the order-one margin of a cold agent needs \(S\sim N\): a collective rearrangement.
Select
Once stranded, it must be the least fit agent of all before it realigns. Warm agents compete for that slot.
Scroll the panel sideways →
Subcritical (left): cascades are small and nothing strands, so gate 1 is shut. Supercritical (right): large cascades strand plenty, but the warm majority always sits deeper at negative fitness and takes the selection, so gate 2 is shut; warm agents are culled and the population cools. Marginal (middle): power-law cascades strand occasionally and the warm pool is small, so a stranded cold agent is often the minimum. The meter compares the cold share of deaths with what the balance needs.
The window closes onto \(\sigma=1\) as \(N\) grows
The same argument can be done in closed form with a two-class caricature: cold agents with margin \(\lambda_0\), and warm agents that carry the cascades. A cascade of \(S\) persistent flips strands a cold agent with probability about \(e^{-S_*/S}\), with \(S_*=\lambda_0^2N/8\). Averaging over the size law of a critical branching (Galton–Watson) process, all dependence on \(N\) and \(\sigma\) collapses into one variable, \(z=(1-\sigma)\sqrt N\), and the supply condition \(\sqrt N\,e^{-\lambda_0 z/2}\gtrsim1\) gives the lower limit of the window:
Above \(\sigma=1\) the other gate shuts: warm agents fill the bottom of the fitness ranking and shield the cold ones from selection. The lower limit is a threshold, not a resting point, and the upper limit is set by shielding, which the argument does not pin down sharply; both converge on 1.
Measured, the evolved population approaches 1 from below: \(\sigma=0.88\) at \(N=128\), saturating at about 1.01 for \(N\ge2048\), within the \(\pm0.015\) bias of our estimator.
At \(N=256\) only about ten warm agents sit above \(T_\times\), and they can fluctuate to nearly none. The population then falls into a degenerate state in which only the newest agent is ever at negative fitness, which satisfies the balance trivially. In \(10^7\)-step runs this happens in 12 of 24 runs at \(N=256\), 2 of 24 at \(N=512\) and none of 8 at \(N=1024\); it explains \(\sigma\approx0.95\) at \(N=256\).
Reached from above in ∼N steps; from below, by aging
Is this state actually reached? We started populations of \(N=1024\) from four different temperature distributions.
The death flux must stay flat however uneven the population; only near σ = 1 can the cold majority keep being renewed.
5 · Why it is optimal
Extremal selection is stability repair at every scale
Why is the critical state also the best one? Each agent's reward is its local stability, and its fitness is a smoothed copy of it (correlation \(r=0.997\)). Replacing the least fit agent therefore means repairing the least stable site. This is the principle of extremal optimization, known to work best at a border between ergodic and non-ergodic behaviour; here that border is self-organized. Small cascades refine the current basin, rare large ones move the population to a new one.
Replacing a random agent instead abolishes the self-organization too (\(\langle T\rangle\approx1.0\)).
Same backbone, half the loss
To see where the gain comes from, split the reward by the sign of each agent's stability:
The backbone gain \(G\) sums the stability of the aligned agents, those playing with their field: how good a configuration the majority holds. The misalignment loss \(L\) sums \(|\lambda_i|\) over the agents playing against their field: the reward lost to instability.
Temperature sets the width \(w=T/2\) of the flip kernel. A wider kernel explores more of the landscape, so \(G\) rises, but it also knocks more agents out of alignment, so \(L\) rises too. With one temperature for everyone, the two are locked together and trace a frontier, labelled by \(T\).
The evolved population (★) matches the frontier's best \(G\) but pays only 52% of its \(L\) (3.2 standard errors). The cold majority holds the backbone at almost no loss; the warm minority does the exploring and is culled before it accumulates misalignment.
Optimality is not a separate finding. It is what marginal, extremal repair does.
6 · Generality
Robust to inheritance; three ingredients suffice
We tried four replacement rules: V0 blank slate (the main model); V1 elitist copying of \(q\) and \(T\) without mutation; V2 fitness-biased inheritance with mutation; V3 Darwinian inheritance of \(T\) only. At \(\alpha=0.75\) the rewards are statistically indistinguishable, and at \(\alpha=0.9\) inheritance improves them. All four collapse onto the same ergodicity transition, with \(T_\times\) in the same window. The balance explains why: a replacement rule only changes the offspring distribution, while \(T_\times\) is defined by the hazard for any rule.
Three general conditions are enough for the mechanism:
Extremal removal
Removal is rank-based: the least fit goes.
Offspring reach the hard class
Some newcomers become established, so the hard class must sustain a death flux.
A hard class
Its members can be removed only after collective rearrangements.
The spin glass and the learning rule only decide which critical point; the mechanism is selection.
7 · Summary
One condition, and the reason evolution meets it
Reward above every fixed \(T\); the evolved state at the \(\epsilon\) edge; avalanches with \(\tau\approx-1.58\).
\(\sigma\simeq\rho(M)=1\): crossings at \(T=0.304\) and \(0.28\), inside the edge window at every \(\alpha\).
Every arrival must die; cold agents need to be stranded (\(\sigma\gtrsim1\)) and selected (\(\sigma\lesssim1\)).
What is exact, what is argued, and what is measured:
- \(\kappa^*=u\), \(\rho^*=u/(Nh)\), unit death flux
- \(q^-=-q^+\); \(\langle R\rangle=-2E/N\); \(R_i=\lambda_i\)
- \(\mathbb{P}[\text{damaged}]=\tfrac12\chi|\delta Q|\) to first order; flip and incentive Jacobians share one spectrum
- \(\sigma\simeq\rho(M)\) assumes a frozen Jacobian over a cascade
- Why marginal: the two-gate argument and the caricature; they bracket the window, not the side of approach
- The avalanche statistic as a proxy for branching-process progeny
- \(T_\times\), \(\sigma=1\) and \(\rho(M)=1\) inside the edge window at four values of \(\alpha\)
- \(\sigma(N)\to\approx1.01\) for \(N\ge2048\)
- Relaxation and aging at \(\alpha=0.75\), 8–24 realisations
Technical note: measuring the branching ratio
We follow the activity \(A(t)\), the number of agents that newly differ at step \(t\), with \(\langle A(t)\rangle\propto\sigma^t\), and fit the ensemble-averaged curve (per-run ratios are biased low where the damage dies). Calibrated on Galton–Watson processes, the bias is at most 0.035 for \(\sigma=0.8\)–\(1.2\) and at most 0.015 near 1. The first step carries a seed transient, \(\langle A(0)\rangle=\alpha\sqrt{2N/\pi}\,\langle\chi\rangle\), from the all-to-one coupling to the seed; including it inflates large-\(N\) estimates to 1.02–1.03, while dropping 4 or 6 initial steps agrees to within 0.005.
Criticality and functionality are not two findings here, but one.