| metric | V2-512 best(step1000) | V2-512 final(step100k) | XL-512 best | XL-512 final(step100k) | |---|---|---|---|---| | 参数 | 4,741,902 | 4,741,902 | 7,898,127 | 7,898,127 | | router_dim | 512 | 512 | 512 | 512 | | 每条记录地址字节 | 2048 | 2048 | 2048 | 2048 | | **Top-1 正确率** | 67.12% | 60.63% | 60.85% | 54.36% | | Recall@1 | 67.12% | 60.63% | 60.85% | 54.36% | | Recall@3 | 88.77% | 85.01% | 92.97% | 85.33% | | Recall@5 | 94.93% | 90.13% | 97.33% | 91.93% | | MRR | 79.08% | 73.26% | 75.98% | 69.50% | | nDCG@3 | 80.00% | 74.57% | 79.28% | 71.71% | | 多跳证据全中(Top-3) | 88.66% | 84.90% | 92.86% | 85.22% | | hop 正确率 | 97.84% | 99.69% | 99.38% | 99.69% | | hop 欠预测率 | 0.33% | 0.33% | 0.33% | 0.33% | | need F1 | 98.92% | 99.89% | 99.62% | 99.89% | | need 召回 | 99.78% | 99.78% | 99.78% | 99.78% | | **未知拒答率** | 66.04% | 100.00% | 90.57% | 100.00% | | **已知问题被误拒率** | 0.22% | 0.22% | 0.22% | 0.22% | | 未知问题被误读率 | 33.96% | 0.00% | 9.43% | 0.00% | | 仲裁准确率 | 97.94% | 99.79% | 99.28% | 99.79% | | 已知问题 need 概率均值 | 98.47% | 99.78% | 99.62% | 99.78% | | 未知问题 need 概率均值 | 25.36% | 1.52% | 13.50% | 1.24% | | 平均分数余量 | 0.5880 | 3.8837 | 6.7097 | 25.3757 | | 单查询延迟 ms(GPU) | 0.8951 | 0.9347 | 1.3102 | 1.2458 | | 路由 QPS(GPU) | 1117.1437 | 1069.9032 | 763.2632 | 802.6816 | ### family = `benchmark_qa` | metric | V2-512 best(step1000) | V2-512 final(step100k) | XL-512 best | XL-512 final(step100k) | |---|---|---|---|---| | episodes | 64 | 64 | 64 | 64 | | Top-1 正确率 | 0.00% | 0.00% | 0.00% | 0.00% | | Recall@3 | 0.00% | 0.00% | 0.00% | 0.00% | | MRR | 0.00% | 0.00% | 0.00% | 0.00% | | hop 正确率 | 100.00% | 100.00% | 100.00% | 100.00% | | 未知拒答率 | 100.00% | 100.00% | 100.00% | 100.00% | | 已知问题被误拒率 | 0.00% | 0.00% | 0.00% | 0.00% | | 未知问题被误读率 | 0.00% | 0.00% | 0.00% | 0.00% | | 平均分数余量 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | ### family = `mega_validation` | metric | V2-512 best(step1000) | V2-512 final(step100k) | XL-512 best | XL-512 final(step100k) | |---|---|---|---|---| | episodes | 20 | 20 | 20 | 20 | | Top-1 正确率 | 40.00% | 35.00% | 40.00% | 50.00% | | Recall@3 | 80.00% | 80.00% | 85.00% | 80.00% | | MRR | 62.26% | 55.24% | 60.83% | 63.61% | | hop 正确率 | 70.00% | 70.00% | 70.00% | 70.00% | | 未知拒答率 | 0.00% | 0.00% | 0.00% | 0.00% | | 已知问题被误拒率 | 20.00% | 20.00% | 20.00% | 20.00% | | 未知问题被误读率 | 0.00% | 0.00% | 0.00% | 0.00% | | 平均分数余量 | -0.2963 | -12.9083 | -0.6136 | 3.7291 | ### family = `memory_policy` | metric | V2-512 best(step1000) | V2-512 final(step100k) | XL-512 best | XL-512 final(step100k) | |---|---|---|---|---| | episodes | 1728 | 1728 | 1728 | 1728 | | Top-1 正确率 | 69.89% | 62.57% | 62.39% | 55.13% | | Recall@3 | 90.04% | 85.94% | 94.20% | 85.76% | | MRR | 81.00% | 74.59% | 77.16% | 70.03% | | hop 正确率 | 98.96% | 100.00% | 99.83% | 100.00% | | 未知拒答率 | 14.29% | 100.00% | 76.19% | 100.00% | | 已知问题被误拒率 | 0.00% | 0.00% | 0.00% | 0.00% | | 未知问题被误读率 | 85.71% | 0.00% | 23.81% | 0.00% | | 平均分数余量 | 0.6516 | 5.3929 | 7.2586 | 27.0634 | ### family = `native_memory` | metric | V2-512 best(step1000) | V2-512 final(step100k) | XL-512 best | XL-512 final(step100k) | |---|---|---|---|---| | episodes | 128 | 128 | 128 | 128 | | Top-1 正确率 | 28.04% | 34.58% | 40.19% | 42.99% | | Recall@3 | 70.09% | 71.03% | 74.77% | 79.44% | | MRR | 51.73% | 55.38% | 59.96% | 62.29% | | hop 正确率 | 85.94% | 100.00% | 97.66% | 100.00% | | 未知拒答率 | 14.29% | 100.00% | 76.19% | 100.00% | | 已知问题被误拒率 | 0.00% | 0.00% | 0.00% | 0.00% | | 未知问题被误读率 | 85.71% | 0.00% | 23.81% | 0.00% | | 平均分数余量 | -0.2618 | -17.0552 | -0.6793 | 2.4967 | ### 读取/拒答门槛曲线 | run | threshold | need F1 | need 召回 | 未知拒答率 | 已知问题被误拒率 | 未知被误读率 | tp/tn/fp/fn | |---|---|---|---|---|---|---|---| | V2-512 best(step1000) | 0.30 | 98.81% | 99.78% | 62.26% | 0.22% | 37.74% | 1830/66/40/4 | | V2-512 best(step1000) | 0.40 | 98.87% | 99.78% | 64.15% | 0.22% | 35.85% | 1830/68/38/4 | | V2-512 best(step1000) | 0.50 | 98.92% | 99.78% | 66.04% | 0.22% | 33.96% | 1830/70/36/4 | | V2-512 best(step1000) | 0.60 | 99.40% | 99.78% | 83.02% | 0.22% | 16.98% | 1830/88/18/4 | | V2-512 best(step1000) | 0.70 | 99.84% | 99.78% | 98.11% | 0.22% | 1.89% | 1830/104/2/4 | | V2-512 final(step100k) | 0.30 | 99.89% | 99.78% | 100.00% | 0.22% | 0.00% | 1830/106/0/4 | | V2-512 final(step100k) | 0.40 | 99.89% | 99.78% | 100.00% | 0.22% | 0.00% | 1830/106/0/4 | | V2-512 final(step100k) | 0.50 | 99.89% | 99.78% | 100.00% | 0.22% | 0.00% | 1830/106/0/4 | | V2-512 final(step100k) | 0.60 | 99.89% | 99.78% | 100.00% | 0.22% | 0.00% | 1830/106/0/4 | | V2-512 final(step100k) | 0.70 | 99.89% | 99.78% | 100.00% | 0.22% | 0.00% | 1830/106/0/4 | | XL-512 best | 0.30 | 99.19% | 99.78% | 75.47% | 0.22% | 24.53% | 1830/80/26/4 | | XL-512 best | 0.40 | 99.51% | 99.78% | 86.79% | 0.22% | 13.21% | 1830/92/14/4 | | XL-512 best | 0.50 | 99.62% | 99.78% | 90.57% | 0.22% | 9.43% | 1830/96/10/4 | | XL-512 best | 0.60 | 99.84% | 99.78% | 98.11% | 0.22% | 1.89% | 1830/104/2/4 | | XL-512 best | 0.70 | 99.89% | 99.78% | 100.00% | 0.22% | 0.00% | 1830/106/0/4 | | XL-512 final(step100k) | 0.30 | 99.89% | 99.78% | 100.00% | 0.22% | 0.00% | 1830/106/0/4 | | XL-512 final(step100k) | 0.40 | 99.89% | 99.78% | 100.00% | 0.22% | 0.00% | 1830/106/0/4 | | XL-512 final(step100k) | 0.50 | 99.89% | 99.78% | 100.00% | 0.22% | 0.00% | 1830/106/0/4 | | XL-512 final(step100k) | 0.60 | 99.89% | 99.78% | 100.00% | 0.22% | 0.00% | 1830/106/0/4 | | XL-512 final(step100k) | 0.70 | 99.89% | 99.78% | 100.00% | 0.22% | 0.00% | 1830/106/0/4 |