STARGA Research · NN_Universe

Universe is a Neural Network: a Fisher-information signature of deep layered computation.

Across 12 parameterised substrate classes, the tier ratio T₁/T₃ of the Fisher Information Matrix diagonal separates into two groups with complete rank separation. The universality class is deep layered sequential composition: not neural networks, not learning, not optimisation.

View the repository Reproduce in one command

github.com/star-ga/nn_universe · STARGA, Inc.

Depth

L = 20

Measured T₁/T₃

9.53 × 10¹⁵

← smaller F_iilog F_ii (centred)larger →

Tier 1 mean ÷ Tier 3 mean, log scale100

Tier 3bottom 50 %
Tier 2middle 49 %
Tier 1top 1 %

Var[log F] = 107.0

Untrained ReLU MLPs, width 64, 5 seeds, 1 000 FIM probes (V6.0 depth sweep). Read-outs are the measured seed means; the curve is a Gaussian model of log F_ii with the measured variance, peak-normalised.

12
parameterised substrate classes: p = 1.7 × 10⁻¹⁷
one-sided Mann–Whitney U: r = 1.000
rank-biserial: 6
classes confirm √L scaling

The observable

Three tiers of the Fisher diagonal.

For each parameter θ_i, the Fisher Information Matrix diagonal F_ii = E[(∂ℓ/∂θ_i)²] records how informative that parameter direction is. Sort the diagonal: the top 1 % is Tier 1, the middle 49 % Tier 2, the bottom 50 % Tier 3. The tier ratio T₁/T₃ divides the mean of Tier 1 by the mean of Tier 3; the dichotomy threshold is T₁/T₃ = 100.

The 1 % / 49 % / 50 % partition is a naming convention. The dichotomy is partition-invariant under three alternative summary statistics: Gini, effective rank and top-1 % mass.

FIM diagonal, sorted ascending

Tier 3
bottom 50 %
Tier 2
middle 49 %
Tier 1
top 1 %

T₁/T₃=mean F_ii in Tier 1mean F_ii in Tier 3

The dichotomy

Twelve substrate classes, two groups, no overlap.

Trained and untrained neural networks, random boolean circuits, four shallow parameterised learners, U(1) and SU(2) lattice gauge fields, three dynamical-system controls and a random-matrix ensemble. Every deep layered sequential observation ranks above every other observation.

T₁/T₃ by substrate class, log scale

deep layered sequential
rest
T₁/T₃ = 100

Random boolean circuits

no learning, no gradients · n = 6

88 440

Neural networks, untrained

Kaiming init, 4 widths · n = 20

3 757

Neural networks, trained

4 widths · n = 20

337

Random matrix ensemble

GOE, N = 3 003 · n = 6

80.7

SU(2) lattice gauge field

non-abelian, 972 params · n = 3

4.85

Cellular automaton

Rule 110, N = 128 · n = 6

3.77

Harmonic chain

N = 128 · n = 6

3.57

Logistic regression

shallow learner · n = 5

3.13

Ising chain

N = 256 · n = 6

2.54

Gaussian process

shallow learner · n = 5

1.97

U(1) lattice gauge field

16 384 link phases · n = 3

1.6

Kernel ridge regression

shallow learner · n = 5

1.42

Linear regression

shallow learner · n = 5

1.10

Bars: 95 % log-bootstrap confidence intervals; ticks: point estimates (printed at right); dots: every per-seed value. Source: experiments/v5_0_dichotomy_stats and the per-seed results it loads.

Data table

Class	Group	Seeds	Point estimate
Random boolean circuits	deep	6	88 440
Neural networks, untrained	deep	20	3 757
Neural networks, trained	deep	20	337
Random matrix ensemble	rest	6	80.7
SU(2) lattice gauge field	rest	3	4.85
Cellular automaton	rest	6	3.77
Harmonic chain	rest	6	3.57
Logistic regression	rest	5	3.13
Ising chain	rest	6	2.54
Gaussian process	rest	5	1.97
U(1) lattice gauge field	rest	3	1.6
Kernel ridge regression	rest	5	1.42
Linear regression	rest	5	1.10

All 96 per-seed observations, ranked by log T₁/T₃one-sided test · unsorted

lowesthighest

Mann–Whitney U

p = 1.7 × 10⁻¹⁷

rank-biserial

r = 1.000

deep observations

n = 46

rest observations

n = 50

The boolean-circuit result is the decisive data point: no neurons, no real-valued weights, no gradients, no probabilistic structure and no training, yet its FIM diagonal hierarchy matches or exceeds a trained ViT.

The mechanism

Depth sets the split: log(T₁/T₃) ∝ √L.

The split is quantitatively predicted by Hanin & Nica 2020 (Comm. Math. Phys. 376, 287–322): log-normal F_ii with depth-linear variance gives log(T₁/T₃) ∝ √L. The scaling is confirmed empirically across six independent substrate classes.

Untrained MLP

R² = 0.98

slope 11.51

x: depth L, √ scale · y: seed-mean log T₁/T₃v6_0_depth_sweep.json

Trained MLP

R² = 0.94

slope 11.78

x: depth L, √ scale · y: seed-mean log T₁/T₃v6_2_trained_depth_sweep.json

Random boolean circuits

R² = 0.98

slope 4.09

x: depth L, √ scale · y: seed-mean log T₁/T₃v6_3_bc_depth_sweep.json

Transformers

R² = 0.97

slope 0.44

x: depth L, √ scale · y: seed-mean log T₁/T₃v6_4_transformer_depth_sweep.json

Binary tensor networks

R² = 0.99

slope 7.44

x: depth L, √ scale · y: seed-mean log T₁/T₃v8_0_bttn_results.json

ResNet residual stacks

R² = 0.999

slope 16.74

x: residual blocks, √ scale · y: seed-mean log T₁/T₃v9_resnet_gpt2_results.json

Across four activations

The mechanism is also confirmed across four activations.

ReLUR² = 0.965
GELUR² = 0.990
SwishR² = 0.990
tanhR² = 0.969

Read the mechanism derivation

Real data

On a real benchmark, the same band.

ResNet-18 trained 10 epochs on CIFAR-10 (81.4 % test accuracy, 11.2 M params) gives T₁/T₃ = 778, Gini = 0.84 and top-1 % FIM mass = 47.8 %, under the same measurement protocol as the synthetic-task panel. Production-scale measurements at ResNet-50 V1+V2 on ImageNet (76.13 / 80.86 % top-1), ViT-L/16 (304 M params), GPT-2-medium, GPT-2-large (774 M) and Pythia-1.4 B all sit firmly in the deep-sequential band.

1 % of parameters, 47.8 % of the Fisher information

ResNet-18 trained 10 epochs on CIFAR-10, FIM diagonal over 200 probes

parameterstop 1 %

Fisher information47.8 %

T₁/T₃

778

Gini

0.84

top-1 % FIM mass

47.8 %

test accuracy

81.4 %

parameters

11.2 M

training

10 epochs

Trained checkpoints, T₁/T₃ on a log scale

ResNet-18
CIFAR-10, 10 epochs
778
778
GPT-2-large
774 M params
1.10 × 10³
1.10 × 10³
Pythia-1.4B
1.4 B params
4.25 × 10³
4.25 × 10³
ViT-L/16
304 M params
6.93 × 10³
6.93 × 10³
GPT-2-medium
5 seeds
log₁₀ = 4.83 ± 0.003
log₁₀ = 4.83 ± 0.003

T₁/T₃ = 100

Scope and limits

Where the scaling stops.

Five falsifiers were registered before measurement: three confirmed the predicted direction, two refuted it and narrowed the scope. GPT-Tiny attention keeps the magnitude of the hierarchy but not the √L scaling; time-unrolled RNN and LSTM do not follow the scaling either.

GPT-Tiny, tied

R² = 0.974

slope −0.22

x: depth L, √ scale · y: seed-mean log T₁/T₃v9_resnet_gpt2_results.json

GPT-Tiny, untied

R² = 0.39

slope −0.027

x: depth L, √ scale · y: seed-mean log T₁/T₃v9_1_gpt_untied_results.json

RNN, time-unrolled

R² = 0.40

slope −0.08

x: sequence length T, √ scale · y: seed-mean log T₁/T₃v9_3_rnn_lstm_results.json

LSTM, time-unrolled

R² = 0.52

slope −0.17

x: sequence length T, √ scale · y: seed-mean log T₁/T₃v9_3_rnn_lstm_results.json

Attention and recurrence
GPT-Tiny attention architectures sit in the deep-sequential band by magnitude but the √L scaling itself does not hold for either tied- or untied-embedding configuration; time-unrolled RNN/LSTM also do not follow the scaling. The mechanism's √L universality is therefore scoped to non-attention layered-stack architectures; the dichotomy magnitude claim is universal across all tested substrates.
Present at initialisation
The hierarchy is already present at random Kaiming initialisation and training decreases it: 4–24× in V4.1, measured at L = 5. At L ≥ 8 the untrained/trained factor collapses to ≈ 1×. Neither learning nor optimisation creates the signature.
Bounded by the panel
“Universality” here is empirical universality across the measured substrate panel. The reverse direction, that any system with T₁/T₃ > 100 at L ≥ 4 is deep-sequential, is conjectural and open.
Cosmology is out of scope
Cosmological and holographic interpretations of the substrate class are out of scope. They are listed for context only; nothing in the results depends on those framings, and the empirical and mechanistic results stand independently.

Reproduce

One command, committed reference outputs.

Wall-clock on a single CPU is ~12 minutes; on a CUDA GPU ~3 minutes. The script runs the load-bearing experiments and writes JSONs alongside the committed reference outputs for byte-level comparison.

--quick skips the V6.0 depth sweep (~7 min savings).

nn_universe

$ python3 -m venv .venv && source .venv/bin/activate$ pip install -r requirements.txt$ bash scripts/reproduce_main_results.sh

$ bash scripts/reproduce_main_results.sh --quick

V5.0 dichotomy stats
experiments/v5_0_dichotomy_stats/dichotomy_stats.py
→ dichotomy_stats_results.json
V5.1 threshold sensitivity
experiments/v5_0_dichotomy_stats/threshold_sensitivity.py
→ v5_1_threshold_sensitivity_results.json
V5.2 multi-seed bootstrap
experiments/v5_0_dichotomy_stats/mw_bootstrap.py
→ v5_2_mw_bootstrap_results.json
V6.0 mechanism (depth sweep)
experiments/v6_0_depth_mechanism/depth_sweep.py
→ v6_0_depth_sweep.json
V4.5 partition-invariant
experiments/v4_3_statistics/partition_invariant_dichotomy.py
→ v4_3_partition_invariant_dichotomy.json
V6.0c pooling-error bound
experiments/v6_0_mechanism/pooling_error_bound.py
→ v6_0c_pooling_error_bound_results.json

Code, data and the full write-up

Experiment scripts with committed JSON outputs, figures and the mechanism notes, in one public repository.

star-ga/nn_universe Documents