| metric | V2-128 v6 final | REPLAY-128 final | |---|---|---| | 参数 | 2,037,774 | 2,037,774 | | router_dim | 128 | 128 | | 每条记录地址字节 | 512 | 512 | | **Top-1 正确率** | 94.37% | 94.14% | | Recall@1 | 88.82% | 88.58% | | Recall@3 | 96.40% | 96.48% | | Recall@5 | 97.40% | 97.15% | | MRR | 95.97% | 95.77% | | nDCG@3 | 95.31% | 95.37% | | 多跳证据全中(Top-3) | 95.95% | 96.22% | | hop 正确率 | 100.00% | 100.00% | | hop 欠预测率 | 0.00% | 0.00% | | need F1 | 100.00% | 100.00% | | need 召回 | 100.00% | 100.00% | | **未知拒答率** | 100.00% | 100.00% | | **已知问题被误拒率** | 0.00% | 0.00% | | 未知问题被误读率 | 0.00% | 0.00% | | 仲裁准确率 | 100.00% | 100.00% | | 已知问题 need 概率均值 | 100.00% | 100.00% | | 未知问题 need 概率均值 | 0.00% | 0.00% | | 平均分数余量 | 51.3153 | 40.1456 | | 单查询延迟 ms(GPU) | 0.8989 | 0.9342 | | 路由 QPS(GPU, 单查询) | 1112.4716 | 1070.4178 | | 批量 QPS(GPU, batch=64) | 67531.9194 | 70303.4031 | | 批量 QPS(GPU, batch=256) | 237234.7328 | 244326.0991 | | 批量延迟 ms(GPU, batch=64) | 0.0148 | 0.0142 | ### family = `benchmark_qa` | metric | V2-128 v6 final | REPLAY-128 final | |---|---|---| | episodes | 64 | 64 | | Top-1 正确率 | - | - | | Recall@3 | - | - | | MRR | - | - | | hop 正确率 | 100.00% | 100.00% | | 未知拒答率 | 100.00% | 100.00% | | 已知问题被误拒率 | - | - | | 未知问题被误读率 | 0.00% | 0.00% | | 平均分数余量 | - | - | ### family = `mega_validation` | metric | V2-128 v6 final | REPLAY-128 final | |---|---|---| | episodes | 20000 | 20000 | | Top-1 正确率 | 98.38% | 98.22% | | Recall@3 | 99.51% | 99.72% | | MRR | 99.19% | 99.11% | | hop 正确率 | 100.00% | 100.00% | | 未知拒答率 | 100.00% | 100.00% | | 已知问题被误拒率 | 0.00% | 0.00% | | 未知问题被误读率 | 0.00% | 0.00% | | 平均分数余量 | 57.1101 | 44.7334 | ### family = `memory_policy` | metric | V2-128 v6 final | REPLAY-128 final | |---|---|---| | episodes | 1728 | 1728 | | Top-1 正确率 | 59.40% | 58.76% | | Recall@3 | 68.42% | 67.31% | | MRR | 67.60% | 66.33% | | hop 正确率 | 100.00% | 100.00% | | 未知拒答率 | 100.00% | 100.00% | | 已知问题被误拒率 | 0.00% | 0.00% | | 未知问题被误读率 | 0.00% | 0.00% | | 平均分数余量 | 0.2715 | -0.2722 | ### family = `native_memory` | metric | V2-128 v6 final | REPLAY-128 final | |---|---|---| | episodes | 128 | 128 | | Top-1 正确率 | 52.34% | 47.66% | | Recall@3 | 77.57% | 77.57% | | MRR | 67.49% | 65.73% | | hop 正确率 | 100.00% | 100.00% | | 未知拒答率 | 100.00% | 100.00% | | 已知问题被误拒率 | 0.00% | 0.00% | | 未知问题被误读率 | 0.00% | 0.00% | | 平均分数余量 | -0.8787 | -1.0887 | ### 读取/拒答门槛曲线 | run | threshold | need F1 | need 召回 | 未知拒答率 | 已知问题被误拒率 | 未知被误读率 | tp/tn/fp/fn | |---|---|---|---|---|---|---|---| | V2-128 v6 final | 0.30 | 100.00% | 100.00% | 100.00% | 0.00% | 0.00% | 17814/4106/0/0 | | V2-128 v6 final | 0.40 | 100.00% | 100.00% | 100.00% | 0.00% | 0.00% | 17814/4106/0/0 | | V2-128 v6 final | 0.50 | 100.00% | 100.00% | 100.00% | 0.00% | 0.00% | 17814/4106/0/0 | | V2-128 v6 final | 0.60 | 100.00% | 100.00% | 100.00% | 0.00% | 0.00% | 17814/4106/0/0 | | V2-128 v6 final | 0.70 | 100.00% | 100.00% | 100.00% | 0.00% | 0.00% | 17814/4106/0/0 | | V2-128 v6 final | 0.80 | 100.00% | 100.00% | 100.00% | 0.00% | 0.00% | 17814/4106/0/0 | | REPLAY-128 final | 0.30 | 100.00% | 100.00% | 100.00% | 0.00% | 0.00% | 17814/4106/0/0 | | REPLAY-128 final | 0.40 | 100.00% | 100.00% | 100.00% | 0.00% | 0.00% | 17814/4106/0/0 | | REPLAY-128 final | 0.50 | 100.00% | 100.00% | 100.00% | 0.00% | 0.00% | 17814/4106/0/0 | | REPLAY-128 final | 0.60 | 100.00% | 100.00% | 100.00% | 0.00% | 0.00% | 17814/4106/0/0 | | REPLAY-128 final | 0.70 | 100.00% | 100.00% | 100.00% | 0.00% | 0.00% | 17814/4106/0/0 | | REPLAY-128 final | 0.80 | 100.00% | 100.00% | 100.00% | 0.00% | 0.00% | 17814/4106/0/0 | ### 按 mega 类别拆解(每类 2,000 条) | 类别 | episodes | metric | V2-128 v6 final | REPLAY-128 final | |---|---|---|---|---| | (未分类) | - | Top-1 正确率 | - | - | | (未分类) | - | Recall@3 | - | - | | (未分类) | - | MRR | - | - | | (未分类) | - | 多跳证据全中 | - | - | | (未分类) | - | hop 正确率 | - | - | | (未分类) | - | 未知拒答率 | - | - | | (未分类) | - | 已知问题被误拒率 | - | - | | (未分类) | - | 未知问题被误读率 | - | - | | conflict_update | 2000 | Top-1 正确率 | 99.30% | 99.10% | | conflict_update | 2000 | Recall@3 | 100.00% | 100.00% | | conflict_update | 2000 | MRR | 99.65% | 99.55% | | conflict_update | 2000 | 多跳证据全中 | 100.00% | 100.00% | | conflict_update | 2000 | hop 正确率 | 100.00% | 100.00% | | conflict_update | 2000 | 未知拒答率 | - | - | | conflict_update | 2000 | 已知问题被误拒率 | 0.00% | 0.00% | | conflict_update | 2000 | 未知问题被误读率 | - | - | | distractor_128 | 2000 | Top-1 正确率 | 97.50% | 97.30% | | distractor_128 | 2000 | Recall@3 | 100.00% | 100.00% | | distractor_128 | 2000 | MRR | 98.75% | 98.65% | | distractor_128 | 2000 | 多跳证据全中 | 100.00% | 100.00% | | distractor_128 | 2000 | hop 正确率 | 100.00% | 100.00% | | distractor_128 | 2000 | 未知拒答率 | - | - | | distractor_128 | 2000 | 已知问题被误拒率 | 0.00% | 0.00% | | distractor_128 | 2000 | 未知问题被误读率 | - | - | | distractor_32 | 2000 | Top-1 正确率 | 98.45% | 98.30% | | distractor_32 | 2000 | Recall@3 | 100.00% | 100.00% | | distractor_32 | 2000 | MRR | 99.22% | 99.15% | | distractor_32 | 2000 | 多跳证据全中 | 100.00% | 100.00% | | distractor_32 | 2000 | hop 正确率 | 100.00% | 100.00% | | distractor_32 | 2000 | 未知拒答率 | - | - | | distractor_32 | 2000 | 已知问题被误拒率 | 0.00% | 0.00% | | distractor_32 | 2000 | 未知问题被误读率 | - | - | | forget_correction | 2000 | Top-1 正确率 | - | - | | forget_correction | 2000 | Recall@3 | - | - | | forget_correction | 2000 | MRR | - | - | | forget_correction | 2000 | 多跳证据全中 | - | - | | forget_correction | 2000 | hop 正确率 | 100.00% | 100.00% | | forget_correction | 2000 | 未知拒答率 | 100.00% | 100.00% | | forget_correction | 2000 | 已知问题被误拒率 | - | - | | forget_correction | 2000 | 未知问题被误读率 | 0.00% | 0.00% | | long_context | 2000 | Top-1 正确率 | 97.95% | 97.80% | | long_context | 2000 | Recall@3 | 100.00% | 100.00% | | long_context | 2000 | MRR | 98.97% | 98.89% | | long_context | 2000 | 多跳证据全中 | 100.00% | 100.00% | | long_context | 2000 | hop 正确率 | 100.00% | 100.00% | | long_context | 2000 | 未知拒答率 | - | - | | long_context | 2000 | 已知问题被误拒率 | 0.00% | 0.00% | | long_context | 2000 | 未知问题被误读率 | - | - | | multi_hop | 2000 | Top-1 正确率 | 98.85% | 99.05% | | multi_hop | 2000 | Recall@3 | 96.05% | 97.72% | | multi_hop | 2000 | MRR | 99.42% | 99.52% | | multi_hop | 2000 | 多跳证据全中 | 92.10% | 95.45% | | multi_hop | 2000 | hop 正确率 | 100.00% | 100.00% | | multi_hop | 2000 | 未知拒答率 | - | - | | multi_hop | 2000 | 已知问题被误拒率 | 0.00% | 0.00% | | multi_hop | 2000 | 未知问题被误读率 | - | - | | paraphrase | 2000 | Top-1 正确率 | 99.50% | 99.40% | | paraphrase | 2000 | Recall@3 | 100.00% | 100.00% | | paraphrase | 2000 | MRR | 99.75% | 99.70% | | paraphrase | 2000 | 多跳证据全中 | 100.00% | 100.00% | | paraphrase | 2000 | hop 正确率 | 100.00% | 100.00% | | paraphrase | 2000 | 未知拒答率 | - | - | | paraphrase | 2000 | 已知问题被误拒率 | 0.00% | 0.00% | | paraphrase | 2000 | 未知问题被误读率 | - | - | | random_position | 2000 | Top-1 正确率 | 98.05% | 97.80% | | random_position | 2000 | Recall@3 | 100.00% | 100.00% | | random_position | 2000 | MRR | 99.02% | 98.89% | | random_position | 2000 | 多跳证据全中 | 100.00% | 100.00% | | random_position | 2000 | hop 正确率 | 100.00% | 100.00% | | random_position | 2000 | 未知拒答率 | - | - | | random_position | 2000 | 已知问题被误拒率 | 0.00% | 0.00% | | random_position | 2000 | 未知问题被误读率 | - | - | | single_fact | 2000 | Top-1 正确率 | 97.45% | 97.05% | | single_fact | 2000 | Recall@3 | 100.00% | 100.00% | | single_fact | 2000 | MRR | 98.72% | 98.52% | | single_fact | 2000 | 多跳证据全中 | 100.00% | 100.00% | | single_fact | 2000 | hop 正确率 | 100.00% | 100.00% | | single_fact | 2000 | 未知拒答率 | - | - | | single_fact | 2000 | 已知问题被误拒率 | 0.00% | 0.00% | | single_fact | 2000 | 未知问题被误读率 | - | - | | unknown | 1920 | Top-1 正确率 | 58.99% | 58.10% | | unknown | 1920 | Recall@3 | 68.96% | 67.92% | | unknown | 1920 | MRR | 67.60% | 66.30% | | unknown | 1920 | 多跳证据全中 | 68.96% | 67.92% | | unknown | 1920 | hop 正确率 | 100.00% | 100.00% | | unknown | 1920 | 未知拒答率 | 100.00% | 100.00% | | unknown | 1920 | 已知问题被误拒率 | 0.00% | 0.00% | | unknown | 1920 | 未知问题被误读率 | 0.00% | 0.00% | | unknown_abstention | 2000 | Top-1 正确率 | - | - | | unknown_abstention | 2000 | Recall@3 | - | - | | unknown_abstention | 2000 | MRR | - | - | | unknown_abstention | 2000 | 多跳证据全中 | - | - | | unknown_abstention | 2000 | hop 正确率 | 100.00% | 100.00% | | unknown_abstention | 2000 | 未知拒答率 | 100.00% | 100.00% | | unknown_abstention | 2000 | 已知问题被误拒率 | - | - | | unknown_abstention | 2000 | 未知问题被误读率 | 0.00% | 0.00% |