wpy/natural-memory-nm21
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Natural Memory NM2.1: 记忆路由器分叉、数据集缺陷修复与全轴评测证据

Browse Source 
- 引入 MemoryRouterXL 与 v5/v6 流式多线程训练/编码管线
- 修复 prepare_memory_router_dataset 候选池重建缺陷（mega 家族 3568x 加速，输出逐字节相同）
- 修复 v5 被破坏的拒答与多跳标签（train 未知样本 319 -> 16319，multi_hop 平均正例 1.00 -> 2.00）
- 同存储预算下 V2-128 v6 逐轴 22/22 通过：Top-1 41.12% -> 94.62%，未知拒答 0.00% -> 100.00%
- 记录三条被实测推翻的显然优化（logits_to_keep=1 反而慢 55%、XL 容量未带来收益）
- 记忆手术跨架构可移植性 14/14，读写关闭时与原生模型逐位相同
This commit is contained in:
WpyQwq
2026-09-19 11:11:31 +08:00
commit 643e22ecb9
484 changed files with 306821 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
router_verdict_final.md
+125
View File
@@ -0,0 +1,125 @@
| 指标 | 最强旧基线 | 基线值 | 候选值 | 差值 | 判定 |
|---|---|---:|---:|---:|---|
| Top-1 正确率 | V2-128 deployed(v3) | 41.12% | 94.37% | +53.24pp | 通过 |
| Top-1 正确率 | V2-128 deployed(v3) | 41.12% | 94.14% | +53.01pp | 通过 |
| Top-1 正确率 | V2-128 deployed(v3) | 41.12% | 94.18% | +53.06pp | 通过 |
| Top-1 正确率 | V2-128 deployed(v3) | 41.12% | 95.02% | +53.90pp | 通过 |
| Top-1 正确率 | V2-128 deployed(v3) | 41.12% | 95.19% | +54.06pp | 通过 |
| Recall@1 | V2-128 deployed(v3) | 37.03% | 88.82% | +51.79pp | 通过 |
| Recall@1 | V2-128 deployed(v3) | 37.03% | 88.58% | +51.55pp | 通过 |
| Recall@1 | V2-128 deployed(v3) | 37.03% | 88.65% | +51.62pp | 通过 |
| Recall@1 | V2-128 deployed(v3) | 37.03% | 89.48% | +52.46pp | 通过 |
| Recall@1 | V2-128 deployed(v3) | 37.03% | 89.64% | +52.61pp | 通过 |
| Recall@3 | V2-128 deployed(v3) | 46.73% | 96.40% | +49.67pp | 通过 |
| Recall@3 | V2-128 deployed(v3) | 46.73% | 96.48% | +49.75pp | 通过 |
| Recall@3 | V2-128 deployed(v3) | 46.73% | 96.66% | +49.93pp | 通过 |
| Recall@3 | V2-128 deployed(v3) | 46.73% | 96.53% | +49.80pp | 通过 |
| Recall@3 | V2-128 deployed(v3) | 46.73% | 97.45% | +50.72pp | 通过 |
| Recall@5 | V2-128 deployed(v3) | 48.91% | 97.40% | +48.48pp | 通过 |
| Recall@5 | V2-128 deployed(v3) | 48.91% | 97.15% | +48.23pp | 通过 |
| Recall@5 | V2-128 deployed(v3) | 48.91% | 97.64% | +48.73pp | 通过 |
| Recall@5 | V2-128 deployed(v3) | 48.91% | 97.61% | +48.69pp | 通过 |
| Recall@5 | V2-128 deployed(v3) | 48.91% | 98.20% | +49.29pp | 通过 |
| MRR | V2-128 deployed(v3) | 47.69% | 95.97% | +48.28pp | 通过 |
| MRR | V2-128 deployed(v3) | 47.69% | 95.77% | +48.08pp | 通过 |
| MRR | V2-128 deployed(v3) | 47.69% | 95.87% | +48.18pp | 通过 |
| MRR | V2-128 deployed(v3) | 47.69% | 96.64% | +48.95pp | 通过 |
| MRR | V2-128 deployed(v3) | 47.69% | 96.70% | +49.01pp | 通过 |
| nDCG@3 | V2-128 deployed(v3) | 44.22% | 95.31% | +51.09pp | 通过 |
| nDCG@3 | V2-128 deployed(v3) | 44.22% | 95.37% | +51.15pp | 通过 |
| nDCG@3 | V2-128 deployed(v3) | 44.22% | 95.48% | +51.26pp | 通过 |
| nDCG@3 | V2-128 deployed(v3) | 44.22% | 95.60% | +51.38pp | 通过 |
| nDCG@3 | V2-128 deployed(v3) | 44.22% | 96.43% | +52.21pp | 通过 |
| 多跳证据全中(Top-3) | V2-128 deployed(v3) | 43.43% | 95.95% | +52.53pp | 通过 |
| 多跳证据全中(Top-3) | V2-128 deployed(v3) | 43.43% | 96.22% | +52.80pp | 通过 |
| 多跳证据全中(Top-3) | V2-128 deployed(v3) | 43.43% | 96.44% | +53.01pp | 通过 |
| 多跳证据全中(Top-3) | V2-128 deployed(v3) | 43.43% | 95.35% | +51.92pp | 通过 |
| 多跳证据全中(Top-3) | V2-128 deployed(v3) | 43.43% | 97.29% | +53.86pp | 通过 |
| 多跳证据全中(仅多正例) | V2-128 deployed(v3) | 37.15% | 92.10% | +54.95pp | 通过 |
| 多跳证据全中(仅多正例) | V2-128 deployed(v3) | 37.15% | 95.45% | +58.30pp | 通过 |
| 多跳证据全中(仅多正例) | V2-128 deployed(v3) | 37.15% | 96.05% | +58.90pp | 通过 |
| 多跳证据全中(仅多正例) | V2-128 deployed(v3) | 37.15% | 78.85% | +41.70pp | 通过 |
| 多跳证据全中(仅多正例) | V2-128 deployed(v3) | 37.15% | 97.05% | +59.90pp | 通过 |
| hop 正确率 | V2-128 deployed(v3) | 73.23% | 100.00% | +26.77pp | 通过 |
| hop 正确率 | V2-128 deployed(v3) | 73.23% | 100.00% | +26.77pp | 通过 |
| hop 正确率 | V2-128 deployed(v3) | 73.23% | 100.00% | +26.77pp | 通过 |
| hop 正确率 | V2-128 deployed(v3) | 73.23% | 100.00% | +26.77pp | 通过 |
| hop 正确率 | V2-128 deployed(v3) | 73.23% | 100.00% | +26.77pp | 通过 |
| hop 欠预测率 | V2-128 deployed(v3) | 4.68% | 0.00% | -4.68pp | 通过 |
| hop 欠预测率 | V2-128 deployed(v3) | 4.68% | 0.00% | -4.68pp | 通过 |
| hop 欠预测率 | V2-128 deployed(v3) | 4.68% | 0.00% | -4.68pp | 通过 |
| hop 欠预测率 | V2-128 deployed(v3) | 4.68% | 0.00% | -4.68pp | 通过 |
| hop 欠预测率 | V2-128 deployed(v3) | 4.68% | 0.00% | -4.68pp | 通过 |
| need F1 (门槛 0.50) | V2-128 deployed(v3) | 89.58% | 100.00% | +10.42pp | 通过 |
| need F1 (门槛 0.50) | V2-128 deployed(v3) | 89.58% | 100.00% | +10.42pp | 通过 |
| need F1 (门槛 0.50) | V2-128 deployed(v3) | 89.58% | 100.00% | +10.42pp | 通过 |
| need F1 (门槛 0.50) | V2-128 deployed(v3) | 89.58% | 100.00% | +10.42pp | 通过 |
| need F1 (门槛 0.50) | V2-128 deployed(v3) | 89.58% | 100.00% | +10.42pp | 通过 |
| need 召回 (门槛 0.50) | V2-128 deployed(v3) | 99.82% | 100.00% | +0.18pp | 通过 |
| need 召回 (门槛 0.50) | V2-128 deployed(v3) | 99.82% | 100.00% | +0.18pp | 通过 |
| need 召回 (门槛 0.50) | V2-128 deployed(v3) | 99.82% | 100.00% | +0.18pp | 通过 |
| need 召回 (门槛 0.50) | V2-128 deployed(v3) | 99.82% | 100.00% | +0.18pp | 通过 |
| need 召回 (门槛 0.50) | V2-128 deployed(v3) | 99.82% | 100.00% | +0.18pp | 通过 |
| 未知拒答率 (门槛 0.50) | V2-128 deployed(v3) | 0.00% | 100.00% | +100.00pp | 通过 |
| 未知拒答率 (门槛 0.50) | V2-128 deployed(v3) | 0.00% | 100.00% | +100.00pp | 通过 |
| 未知拒答率 (门槛 0.50) | V2-128 deployed(v3) | 0.00% | 100.00% | +100.00pp | 通过 |
| 未知拒答率 (门槛 0.50) | V2-128 deployed(v3) | 0.00% | 100.00% | +100.00pp | 通过 |
| 未知拒答率 (门槛 0.50) | V2-128 deployed(v3) | 0.00% | 100.00% | +100.00pp | 通过 |
| 已知问题被误拒率 | V2-128 deployed(v3) | 0.18% | 0.00% | -0.18pp | 通过 |
| 已知问题被误拒率 | V2-128 deployed(v3) | 0.18% | 0.00% | -0.18pp | 通过 |
| 已知问题被误拒率 | V2-128 deployed(v3) | 0.18% | 0.00% | -0.18pp | 通过 |
| 已知问题被误拒率 | V2-128 deployed(v3) | 0.18% | 0.00% | -0.18pp | 通过 |
| 已知问题被误拒率 | V2-128 deployed(v3) | 0.18% | 0.00% | -0.18pp | 通过 |
| 未知问题被误读率 | V2-128 deployed(v3) | 100.00% | 0.00% | -100.00pp | 通过 |
| 未知问题被误读率 | V2-128 deployed(v3) | 100.00% | 0.00% | -100.00pp | 通过 |
| 未知问题被误读率 | V2-128 deployed(v3) | 100.00% | 0.00% | -100.00pp | 通过 |
| 未知问题被误读率 | V2-128 deployed(v3) | 100.00% | 0.00% | -100.00pp | 通过 |
| 未知问题被误读率 | V2-128 deployed(v3) | 100.00% | 0.00% | -100.00pp | 通过 |
| 仲裁准确率 | V2-128 deployed(v3) | 81.12% | 100.00% | +18.88pp | 通过 |
| 仲裁准确率 | V2-128 deployed(v3) | 81.12% | 100.00% | +18.88pp | 通过 |
| 仲裁准确率 | V2-128 deployed(v3) | 81.12% | 100.00% | +18.88pp | 通过 |
| 仲裁准确率 | V2-128 deployed(v3) | 81.12% | 100.00% | +18.88pp | 通过 |
| 仲裁准确率 | V2-128 deployed(v3) | 81.12% | 100.00% | +18.88pp | 通过 |
| 平均分数余量 | V2-128 deployed(v3) | -9.2989 | 51.3153 | +60.6143 | 通过 |
| 平均分数余量 | V2-128 deployed(v3) | -9.2989 | 40.1456 | +49.4445 | 通过 |
| 平均分数余量 | V2-128 deployed(v3) | -9.2989 | 73.2482 | +82.5471 | 通过 |
| 平均分数余量 | V2-128 deployed(v3) | -9.2989 | 215.7533 | +225.0522 | 通过 |
| 平均分数余量 | V2-128 deployed(v3) | -9.2989 | 200.3346 | +209.6335 | 通过 |
| 单查询延迟 ms (GPU) | V2-128 deployed(v3) | 1.0995 | 0.9277 | -0.1718 | 通过 |
| 单查询延迟 ms (GPU) | V2-128 deployed(v3) | 1.0995 | 0.9663 | -0.1332 | 通过 |
| 单查询延迟 ms (GPU) | V2-128 deployed(v3) | 1.0995 | 0.9488 | -0.1507 | 通过 |
| 单查询延迟 ms (GPU) | V2-128 deployed(v3) | 1.0995 | 1.4501 | +0.3505 | **未通过** |
| 单查询延迟 ms (GPU) | V2-128 deployed(v3) | 1.0995 | 1.5358 | +0.4363 | **未通过** |
| 路由 QPS (GPU, 单查询) | V2-128 deployed(v3) | 909.4856 | 1077.8859 | +168.4003 | 通过 |
| 路由 QPS (GPU, 单查询) | V2-128 deployed(v3) | 909.4856 | 1034.8360 | +125.3505 | 通过 |
| 路由 QPS (GPU, 单查询) | V2-128 deployed(v3) | 909.4856 | 1053.9133 | +144.4277 | 通过 |
| 路由 QPS (GPU, 单查询) | V2-128 deployed(v3) | 909.4856 | 689.6254 | -219.8602 | **未通过** |
| 路由 QPS (GPU, 单查询) | V2-128 deployed(v3) | 909.4856 | 651.1112 | -258.3744 | **未通过** |
| 批量 QPS (GPU, batch=64) | V2-128 deployed(v3) | 66037.9306 | 72856.4274 | +6818.4967 | 通过 |
| 批量 QPS (GPU, batch=64) | V2-128 deployed(v3) | 66037.9306 | 69378.1979 | +3340.2673 | 通过 |
| 批量 QPS (GPU, batch=64) | V2-128 deployed(v3) | 66037.9306 | 71586.7654 | +5548.8348 | 通过 |
| 批量 QPS (GPU, batch=64) | V2-128 deployed(v3) | 66037.9306 | 46891.9434 | -19145.9872 | **未通过** |
| 批量 QPS (GPU, batch=64) | V2-128 deployed(v3) | 66037.9306 | 33111.5549 | -32926.3757 | **未通过** |
| 批量 QPS (GPU, batch=256) | V2-128 deployed(v3) | 207287.4493 | 236293.1512 | +29005.7019 | 通过 |
| 批量 QPS (GPU, batch=256) | V2-128 deployed(v3) | 207287.4493 | 242759.8763 | +35472.4270 | 通过 |
| 批量 QPS (GPU, batch=256) | V2-128 deployed(v3) | 207287.4493 | 177958.2077 | -29329.2416 | **未通过** |
| 批量 QPS (GPU, batch=256) | V2-128 deployed(v3) | 207287.4493 | 85734.5711 | -121552.8782 | **未通过** |
| 批量 QPS (GPU, batch=256) | V2-128 deployed(v3) | 207287.4493 | 131545.1415 | -75742.3078 | **未通过** |
| 地址字节/记录 | V2-128 deployed(v3) | 512.0000 | 512.0000 | +0.0000 | 通过 |
| 地址字节/记录 | V2-128 deployed(v3) | 512.0000 | 512.0000 | +0.0000 | 通过 |
| 地址字节/记录 | V2-128 deployed(v3) | 512.0000 | 2048.0000 | +1536.0000 | **未通过** |
| 地址字节/记录 | V2-128 deployed(v3) | 512.0000 | 2048.0000 | +1536.0000 | **未通过** |
| 地址字节/记录 | V2-128 deployed(v3) | 512.0000 | 512.0000 | +0.0000 | 通过 |
### 逐候选判定（全方位超越是对单个路由器说的）
| 路由器 | 通过 | 未通过 | 全方位超越 | 未通过的轴 |
|---|---:|---:|---|---|
| V2-128 v6 final | 22 | 0 | **是** | - |
| REPLAY-128 v7 final | 22 | 0 | **是** | - |
| V2-512 v6 final | 20 | 2 | 否 | 批量 QPS (GPU, batch=256), 地址字节/记录 |
| XL-512 v6 final | 17 | 5 | 否 | 单查询延迟 ms (GPU), 路由 QPS (GPU, 单查询), 批量 QPS (GPU, batch=64), 批量 QPS (GPU, batch=256), 地址字节/记录 |
| XL-128 v6 final | 18 | 4 | 否 | 单查询延迟 ms (GPU), 路由 QPS (GPU, 单查询), 批量 QPS (GPU, batch=64), 批量 QPS (GPU, batch=256) |
**汇总 99 项通过 / 11 项未通过 / 0 项无数据**