WpyQwq
643e22ecb9
- 引入 MemoryRouterXL 与 v5/v6 流式多线程训练/编码管线 - 修复 prepare_memory_router_dataset 候选池重建缺陷（mega 家族 3568x 加速，输出逐字节相同） - 修复 v5 被破坏的拒答与多跳标签（train 未知样本 319 -> 16319，multi_hop 平均正例 1.00 -> 2.00） - 同存储预算下 V2-128 v6 逐轴 22/22 通过：Top-1 41.12% -> 94.62%，未知拒答 0.00% -> 100.00% - 记录三条被实测推翻的显然优化（logits_to_keep=1 反而慢 55%、XL 容量未带来收益） - 记忆手术跨架构可移植性 14/14，读写关闭时与原生模型逐位相同
65 lines
3.1 KiB
Markdown
65 lines
3.1 KiB
Markdown
# 零重叠改写（zero-overlap paraphrase）阶段结果
|
||
|
||
所有速率均为百分比。数据来源为各阶段保存的 scorecard JSON，不引用滚动日志。
|
||
|
||
## 零重叠评测集（300 条，24 个**未见过的**改写问法，24 个同形候选，随机猜测 4.17%）
|
||
|
||
`基线` 含部署态路由器与 v6；`冻结策略重放` 为合并数据重放训练后的路由器。
|
||
|
||
| 指标 | V2-128 v6 best | V2-128 v6 final | V2-128 deployed(v3) | ZOV-128 final | REPLAY-128 final |
|
||
|---|---|---|---|---|---|
|
||
| Top-1 正确率 | 7.60% | 18.40% | 11.60% | 66.80% | 59.60% |
|
||
| Recall@3 | 23.60% | 36.00% | 32.00% | 84.40% | 83.20% |
|
||
| MRR | 22.88% | 35.07% | 29.48% | 76.95% | 73.06% |
|
||
| 多跳证据全中(Top-3) | 23.60% | 36.00% | 32.00% | 84.40% | 83.20% |
|
||
| hop 正确率 | 83.33% | 83.33% | 62.33% | 58.33% | 83.33% |
|
||
| 未知拒答率 | 0.00% | 0.00% | 6.00% | 40.00% | 0.00% |
|
||
| 已知问题被误拒率 | 0.00% | 0.00% | 15.20% | 38.00% | 0.00% |
|
||
| 未知问题被误读率 | 100.00% | 100.00% | 94.00% | 60.00% | 100.00% |
|
||
| 每条记录地址字节 | 512 | 512 | 512 | 512 | 512 |
|
||
| 参数量 | 2,037,774 | 2,037,774 | 2,037,774 | 2,037,774 | 2,037,774 |
|
||
| 单查询延迟中位数 ms (GPU) | 0.8446 | 0.8527 | 0.9079 | 0.8515 | 0.8460 |
|
||
| 路由 QPS (GPU, 单查询) | 1153.8630 | 1124.9066 | 1086.5613 | 1128.3880 | 1148.2401 |
|
||
| 批量 QPS (GPU, batch=64) | 61288.5926 | 75940.9559 | 71565.9526 | 76974.8870 | 72823.2671 |
|
||
| 批量 QPS (GPU, batch=256) | 273229.8761 | 296056.4359 | 257898.1302 | 291924.1911 | 234140.6307 |
|
||
|
||
## v6 评测集（21,920 条，全部 22 轴）
|
||
|
||
`基线` 同时包含朴素微调（仅零重叠数据）以显示灾难性遗忘，以及 v6 原始路由器。
|
||
|
||
| 指标 | V2-128 v6 final | ZOV-128 final | REPLAY-128 final |
|
||
|---|---|---|---|
|
||
| Top-1 正确率 | 94.37% | 90.72% | 94.14% |
|
||
| Recall@3 | 96.40% | 93.69% | 96.48% |
|
||
| MRR | 95.97% | 93.50% | 95.77% |
|
||
| 多跳证据全中(Top-3) | 95.95% | 91.96% | 96.22% |
|
||
| hop 正确率 | 100.00% | 91.59% | 100.00% |
|
||
| 未知拒答率 | 100.00% | 100.00% | 100.00% |
|
||
| 已知问题被误拒率 | 0.00% | 13.85% | 0.00% |
|
||
| 未知问题被误读率 | 0.00% | 0.00% | 0.00% |
|
||
| 每条记录地址字节 | 512 | 512 | 512 |
|
||
| 参数量 | 2,037,774 | 2,037,774 | 2,037,774 |
|
||
| 单查询延迟中位数 ms (GPU) | 0.9020 | 0.9837 | 0.8828 |
|
||
| 路由 QPS (GPU, 单查询) | 1031.8867 | 968.8902 | 1070.4178 |
|
||
| 批量 QPS (GPU, batch=64) | 77868.3539 | 63339.9972 | 70303.4031 |
|
||
| 批量 QPS (GPU, batch=256) | 275832.3455 | 251226.6928 | 244326.0991 |
|
||
|
||
### 基线 — 分类别 Top-1
|
||
|
||
| 类别 | V2-128 v6 final | ZOV-128 final | REPLAY-128 final |
|
||
|---|---|---|---|
|
||
| benchmark_qa | - | - | - |
|
||
| mega_validation | 98.38% | 95.61% | - |
|
||
| memory_policy | 59.40% | 49.68% | - |
|
||
| native_memory | 52.34% | 13.08% | - |
|
||
|
||
### 冻结策略重放 — 分类别 Top-1
|
||
|
||
| 类别 | V2-128 v6 final | ZOV-128 final | REPLAY-128 final |
|
||
|---|---|---|---|
|
||
| benchmark_qa | - | - | - |
|
||
| mega_validation | 98.38% | - | 98.22% |
|
||
| memory_policy | 59.40% | - | 58.76% |
|
||
| native_memory | 52.34% | - | 47.66% |
|
||
|