The pre-registered GNN-1 verdict is NO_DIFFERENCE

# The pre-registered GNN-1 verdict is NO_DIFFERENCE

**QMAI Research · 2026-11-02**

## The experiment verdict

Experiment GNN-1 is reported as recorded. The headline is the pre-registered experiment verdict.¹

That verdict is NO_DIFFERENCE. The recorded rule is "unweighted mean of included pairs, resampled then averaged". Text-GCN AUC minus untrained text-cosine AUC has mean 0.072784. The 95 percent interval is -0.028017 to 0.172305. The lower end is not above 0, and the upper end is not below 0, so the frozen rule names NO_DIFFERENCE. The experiment includes 2 pairs.²

In plain terms: across the two pairs, a graph model that reads node text was not clearly better than simply comparing the two nodes' text.

Not an Aurion product claim.

## The other verdicts, at the same rank

These three labels have the same rank. The gate+gate win is not the headline.³

On gate+gate the pair verdict is GNN_BETTER. Text-GCN test AUC is 0.985731. Untrained cosine test AUC is 0.887039. The difference is 0.098692, and the 95 percent interval is 0.026159 to 0.184304. The lower end is above 0. This pair label does not replace the experiment mean.⁴

On doc+memory the pair verdict is NO_DIFFERENCE. Text-GCN test AUC is 0.8125. Untrained cosine test AUC is 0.765625. The difference is 0.046875, and the 95 percent interval is -0.131944 to 0.220486.⁵

The diagnostic verdict is NO_DIFFERENCE, so message passing is not shown to add value. Text-GCN minus the identity-adjacency MLP has mean -0.051346. The 95 percent interval is -0.107591 to 0.005993. The diagnostic includes 2 pairs. At this same rank, the doc+memory diagnostic row is MLP_BETTER, difference -0.114583, interval -0.223958 to -0.001736. On doc+memory, the version without message passing scored higher. The gate+gate diagnostic row is NO_DIFFERENCE, difference 0.011891, interval -0.011891 to 0.045184. Each pair diagnostic note says it is not the primary verdict.⁶

## The question

The question fixed beforehand is whether, inside one node-type pair, a text GCN beats a text cosine that is not trained. Negatives are type-matched. Neighbour scores and the type prior are printed and do not decide.⁷

## The design that was frozen

The input is the public file mindweb-graph.json, sha256 d1123063f55edf2b2da93460683f1b6ea6e9fb122a671119928d3f27893572e2, 3880614 bytes. The result records input.note as null. The graph has 3501 nodes and 629 usable undirected edges. The scored pairs are gate+gate, with 291 edges, and doc+memory, with 240 edges.⁸

Features are onehot(node type in vocab order) + constant + text-64. The encoder has 2 layers, widths 32 and 16, and an inner-product decoder. Training runs for 200 epochs. There is no validation split. Test positives are 29 on gate+gate and 24 on doc+memory, and the test negatives are 29 and 24. Training negatives never include a test pair; the code was fixed and cross-family reviewed before the run. The run plan says not to change parameters and re-run. This note reports that one recorded run.⁹

## What this run does not show

The scored set is a thin set of 2 pairs. One seed and a random split means each interval covers that pair's test rows only. A GNN_BETTER label on gate+gate does not set the experiment verdict. The diagnostic does not show that message passing adds value. The result has no node ids and no node text. This note is not a peer-reviewed paper.¹⁰

Not an Aurion product claim.

## Reproducibility

The stored pre-registration hash is 28879d8d5719a591d46f373534a0efb389e6029ff0097b59e278c08641167857. The deterministic hash is ac2e8138d8d71897dbb6c4a1340b2e7d6e3623bd4e3d704cfdd29a09b36c727e. The recorded runtime is Python 3.12.13 on Linux 6.8.0-134-generic x86_64, and total_s is 63.485.¹¹

## Sources

1. RESULT.json key experiment. The value is GNN-1.
2. RESULT.json keys experiment_verdict.verdict, experiment_verdict.rule, experiment_verdict.auc_diff_mean, experiment_verdict.ci95, and experiment_verdict.included_pairs. The decision rule is in PREREG_GNN1.md. The line is recorded in SOURCES.md.
3. The task ruling: the other verdicts have equal rank, and the gate+gate win is not the headline.
4. RESULT.json scored_pairs entry 1, pair gate then gate: primary.verdict, primary.auc_gcn, primary.auc_cosine, primary.auc_diff, and primary.ci95.
5. RESULT.json scored_pairs entry 0, pair doc then memory: primary.verdict, primary.auc_gcn, primary.auc_cosine, primary.auc_diff, and primary.ci95.
6. RESULT.json keys diagnostic_verdict.verdict, diagnostic_verdict.auc_diff_mean, diagnostic_verdict.ci95, and diagnostic_verdict.included_pairs. Pair rows: scored_pairs entry 0 diagnostic.verdict, diagnostic.auc_diff, diagnostic.ci95, and diagnostic.note; scored_pairs entry 1 diagnostic.verdict, diagnostic.auc_diff, diagnostic.ci95, and diagnostic.note.
7. PREREG_GNN1.md states the within-pair question, the type-matched negatives, the untrained cosine, and that neighbour scores and the type prior do not decide. Lines are in SOURCES.md.
8. RESULT.json keys input.sha256, input.bytes, input.note, input.basename, graph_stats.nodes, and graph_stats.edges_usable_undirected. Pair edge counts: scored_pairs entry 1 edges, and scored_pairs entry 0 edges. The same sha256 and byte count are the pre-registration pin. The line is in SOURCES.md.
9. RESULT.json keys hyperparameters.features, hyperparameters.layers, hyperparameters.hidden1, hyperparameters.hidden2, hyperparameters.decoder, hyperparameters.epochs, hyperparameters.split, and hyperparameters.train_negatives. Test sizes: scored_pairs entry 1 split.test_pos and split.test_neg, and scored_pairs entry 0 split.test_pos and split.test_neg. The no-rerun sentence is in RUN_PLAN.md. The line is in SOURCES.md.
10. RESULT.json key experiment_verdict.included_pairs. PREREG_GNN1.md states that only two pairs meet the edge bar, and that one seed and a random split are limits. Lines are in SOURCES.md. The result file contains no node text fields.
11. RESULT.json keys prereg_sha256, deterministic_sha256, runtime.python, runtime.platform, and runtime.total_s.