wpy/natural-memory-nm21
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Natural Memory NM2.1: 记忆路由器分叉、数据集缺陷修复与全轴评测证据

Browse Source 
- 引入 MemoryRouterXL 与 v5/v6 流式多线程训练/编码管线
- 修复 prepare_memory_router_dataset 候选池重建缺陷（mega 家族 3568x 加速，输出逐字节相同）
- 修复 v5 被破坏的拒答与多跳标签（train 未知样本 319 -> 16319，multi_hop 平均正例 1.00 -> 2.00）
- 同存储预算下 V2-128 v6 逐轴 22/22 通过：Top-1 41.12% -> 94.62%，未知拒答 0.00% -> 100.00%
- 记录三条被实测推翻的显然优化（logits_to_keep=1 反而慢 55%、XL 容量未带来收益）
- 记忆手术跨架构可移植性 14/14，读写关闭时与原生模型逐位相同
This commit is contained in:
WpyQwq
2026-09-19 11:11:31 +08:00
commit 643e22ecb9
484 changed files with 306821 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
replay_check_zov.md
+86
View File
@@ -0,0 +1,86 @@
| metric | V2-128 v6 final | ZOV-128 final | REPLAY-128 final |
|---|---|---|---|
| 参数 | 2,037,774 | 2,037,774 | 2,037,774 |
| router_dim | 128 | 128 | 128 |
| 每条记录地址字节 | 512 | 512 | 512 |
| **Top-1 正确率** | 18.40% | 66.80% | 59.60% |
| Recall@1 | 18.40% | 66.80% | 59.60% |
| Recall@3 | 36.00% | 84.40% | 83.20% |
| Recall@5 | 60.40% | 90.40% | 90.80% |
| MRR | 35.07% | 76.95% | 73.06% |
| nDCG@3 | 28.40% | 76.96% | 73.44% |
| 多跳证据全中(Top-3) | 36.00% | 84.40% | 83.20% |
| hop 正确率 | 83.33% | 58.33% | 83.33% |
| hop 欠预测率 | 0.00% | 38.00% | 0.00% |
| need F1 | 90.91% | 71.26% | 90.91% |
| need 召回 | 100.00% | 62.00% | 100.00% |
| **未知拒答率** | 0.00% | 40.00% | 0.00% |
| **已知问题被误拒率** | 0.00% | 38.00% | 0.00% |
| 未知问题被误读率 | 100.00% | 60.00% | 100.00% |
| 仲裁准确率 | 83.33% | 58.33% | 83.33% |
| 已知问题 need 概率均值 | 99.18% | 66.88% | 99.18% |
| 未知问题 need 概率均值 | 98.22% | 64.00% | 98.22% |
| 平均分数余量 | -18.6882 | 3.1264 | 1.5204 |
| 单查询延迟 ms(GPU) | 0.9001 | 0.8862 | 0.8709 |
| 路由 QPS(GPU, 单查询) | 1111.0124 | 1128.3880 | 1148.2401 |
| 批量 QPS(GPU, batch=64) | 63459.3265 | 76974.8870 | 72823.2671 |
| 批量 QPS(GPU, batch=256) | 250582.4084 | 291924.1911 | 234140.6307 |
| 批量延迟 ms(GPU, batch=64) | 0.0158 | 0.0130 | 0.0137 |
### family = `zero_overlap_paraphrase`
| metric | V2-128 v6 final | ZOV-128 final | REPLAY-128 final |
|---|---|---|---|
| episodes | 300 | 300 | 300 |
| Top-1 正确率 | 18.40% | 66.80% | 59.60% |
| Recall@3 | 36.00% | 84.40% | 83.20% |
| MRR | 35.07% | 76.95% | 73.06% |
| hop 正确率 | 83.33% | 58.33% | 83.33% |
| 未知拒答率 | 0.00% | 40.00% | 0.00% |
| 已知问题被误拒率 | 0.00% | 38.00% | 0.00% |
| 未知问题被误读率 | 100.00% | 60.00% | 100.00% |
| 平均分数余量 | -18.6882 | 3.1264 | 1.5204 |
### 读取/拒答门槛曲线
| run | threshold | need F1 | need 召回 | 未知拒答率 | 已知问题被误拒率 | 未知被误读率 | tp/tn/fp/fn |
|---|---|---|---|---|---|---|---|
| V2-128 v6 final | 0.30 | 90.91% | 100.00% | 0.00% | 0.00% | 100.00% | 250/0/50/0 |
| V2-128 v6 final | 0.40 | 90.91% | 100.00% | 0.00% | 0.00% | 100.00% | 250/0/50/0 |
| V2-128 v6 final | 0.50 | 90.91% | 100.00% | 0.00% | 0.00% | 100.00% | 250/0/50/0 |
| V2-128 v6 final | 0.60 | 90.91% | 100.00% | 0.00% | 0.00% | 100.00% | 250/0/50/0 |
| V2-128 v6 final | 0.70 | 90.91% | 100.00% | 0.00% | 0.00% | 100.00% | 250/0/50/0 |
| V2-128 v6 final | 0.80 | 90.91% | 100.00% | 0.00% | 0.00% | 100.00% | 250/0/50/0 |
| ZOV-128 final | 0.30 | 84.40% | 84.40% | 22.00% | 15.60% | 78.00% | 211/11/39/39 |
| ZOV-128 final | 0.40 | 80.33% | 76.80% | 28.00% | 23.20% | 72.00% | 192/14/36/58 |
| ZOV-128 final | 0.50 | 71.26% | 62.00% | 40.00% | 38.00% | 60.00% | 155/20/30/95 |
| ZOV-128 final | 0.60 | 71.26% | 62.00% | 40.00% | 38.00% | 60.00% | 155/20/30/95 |
| ZOV-128 final | 0.70 | 71.26% | 62.00% | 40.00% | 38.00% | 60.00% | 155/20/30/95 |
| ZOV-128 final | 0.80 | 65.20% | 53.20% | 50.00% | 46.80% | 50.00% | 133/25/25/117 |
| REPLAY-128 final | 0.30 | 90.91% | 100.00% | 0.00% | 0.00% | 100.00% | 250/0/50/0 |
| REPLAY-128 final | 0.40 | 90.91% | 100.00% | 0.00% | 0.00% | 100.00% | 250/0/50/0 |
| REPLAY-128 final | 0.50 | 90.91% | 100.00% | 0.00% | 0.00% | 100.00% | 250/0/50/0 |
| REPLAY-128 final | 0.60 | 90.91% | 100.00% | 0.00% | 0.00% | 100.00% | 250/0/50/0 |
| REPLAY-128 final | 0.70 | 90.91% | 100.00% | 0.00% | 0.00% | 100.00% | 250/0/50/0 |
| REPLAY-128 final | 0.80 | 90.91% | 100.00% | 0.00% | 0.00% | 100.00% | 250/0/50/0 |
### 按 mega 类别拆解（每类 2,000 条）
| 类别 | episodes | metric | V2-128 v6 final | ZOV-128 final | REPLAY-128 final |
|---|---|---|---|---|---|
| (未分类) | - | Top-1 正确率 | - | - | - |
| (未分类) | - | Recall@3 | - | - | - |
| (未分类) | - | MRR | - | - | - |
| (未分类) | - | 多跳证据全中 | - | - | - |
| (未分类) | - | hop 正确率 | - | - | - |
| (未分类) | - | 未知拒答率 | - | - | - |
| (未分类) | - | 已知问题被误拒率 | - | - | - |
| (未分类) | - | 未知问题被误读率 | - | - | - |
| zero_overlap_paraphrase | 300 | Top-1 正确率 | 18.40% | 66.80% | 59.60% |
| zero_overlap_paraphrase | 300 | Recall@3 | 36.00% | 84.40% | 83.20% |
| zero_overlap_paraphrase | 300 | MRR | 35.07% | 76.95% | 73.06% |
| zero_overlap_paraphrase | 300 | 多跳证据全中 | 36.00% | 84.40% | 83.20% |
| zero_overlap_paraphrase | 300 | hop 正确率 | 83.33% | 58.33% | 83.33% |
| zero_overlap_paraphrase | 300 | 未知拒答率 | 0.00% | 40.00% | 0.00% |
| zero_overlap_paraphrase | 300 | 已知问题被误拒率 | 0.00% | 38.00% | 0.00% |
| zero_overlap_paraphrase | 300 | 未知问题被误读率 | 100.00% | 60.00% | 100.00% |