wpy/natural-memory-nm21
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
natural-memory-nm21/rv2_order_current.json
T
WpyQwq 643e22ecb9 Natural Memory NM2.1: 记忆路由器分叉、数据集缺陷修复与全轴评测证据 
- 引入 MemoryRouterXL 与 v5/v6 流式多线程训练/编码管线
- 修复 prepare_memory_router_dataset 候选池重建缺陷（mega 家族 3568x 加速，输出逐字节相同）
- 修复 v5 被破坏的拒答与多跳标签（train 未知样本 319 -> 16319，multi_hop 平均正例 1.00 -> 2.00）
- 同存储预算下 V2-128 v6 逐轴 22/22 通过：Top-1 41.12% -> 94.62%，未知拒答 0.00% -> 100.00%
- 记录三条被实测推翻的显然优化（logits_to_keep=1 反而慢 55%、XL 容量未带来收益）
- 记忆手术跨架构可移植性 14/14，读写关闭时与原生模型逐位相同
2026-09-19 11:11:31 +08:00

3244 lines
107 KiB
JSON
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
{
"eval_file": "data/realistic_v2/eval.jsonl",
"episodes": 200,
"candidate_count": 5,
"router": "checkpoints\\router_replay_v7_v2_128\\memory_router_v2.pt",
"results": {
"prior=1.00 blend=0.00": {
"router": "prior=1.00 blend=0.00",
"cases": 200,
"answerable_cases": 175,
"unknown_cases": 25,
"accuracy_pct": 76.57142857142857,
"wrong_attribute_pct": 17.714285714285715,
"unknown_leak_pct": 56.0,
"mean_records_selected": 4.025,
"records_active_min": 2,
"records_active_max": 9,
"stop_reasons": {
"evidence_found": 200
},
"top_score_answerable": {
"n": 175,
"min": 1.61,
"p10": 2.91,
"p50": 8.37,
"p90": 8.52,
"max": 8.54
},
"top_score_unknown": {
"n": 25,
"min": 1.67,
"p10": 1.68,
"p50": 2.94,
"p90": 8.52,
"max": 8.54
},
"seconds": 435.7
}
},
"rows": {
"prior=1.00 blend=0.00": [
{
"id": "rv2-multi_hop-000147",
"attribute": "",
"answerable": true,
"expected": "值班人-3615",
"reply": "负责该项目的同事对应的值班人是 值班人 -3615。编号 REF-KW5MPC 对应的值班人是 值班人 -3615。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-near_miss-000196",
"attribute": "手机尾号",
"answerable": true,
"expected": "v2.8.0",
"reply": "您的设备末几位数字分别是：user；手机尾号；v2.8.0；座机尾号；256GB；代码仓库；李工 139****3344。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 2.9743022915967714,
"need_memory": true
},
{
"id": "rv2-long_fact-000070",
"attribute": "",
"answerable": true,
"expected": "10.20.3.7:5432",
"reply": "关于培训周期，之前散会时临时定了下来，当时讨论得比较久，最后确认10.20.3.7:5432，后续如果还有调整会在群里同步，暂时先按这个执行",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.370299948573528,
"need_memory": true
},
{
"id": "rv2-noise_context-000181",
"attribute": "",
"answerable": true,
"expected": "10.30.7.3:6379",
"reply": "您提交到 N-V2YMQAFZ。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 8.367098528222034,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000113",
"attribute": "",
"answerable": true,
"expected": "32GB",
"reply": "我常年落脚在 32GB。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 1.6141630959510804,
"need_memory": true
},
{
"id": "rv2-near_miss-000084",
"attribute": "办公城市",
"answerable": true,
"expected": "91.5%",
"reply": "您的办公城市是 91.5%。user；办公城市；91.5%",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 3.8006438164160192,
"need_memory": true
},
{
"id": "rv2-update_conflict-000106",
"attribute": "发布窗口",
"answerable": true,
"expected": "10.30.1.9:5432",
"reply": "关于变更时段，当前长期记忆中未包含具体信息，证据不足，明确说不知道。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 8.518243909577894,
"need_memory": true
},
{
"id": "rv2-multi_hop-000099",
"attribute": "",
"answerable": true,
"expected": "值班人-4408",
"reply": "负责该项目的同事对应的值班人是 值班人-4408。编号 REF-786KY7 对应的值班人是 值班人-4408。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000145",
"attribute": "",
"answerable": true,
"expected": "v1.9.7",
"reply": "我周末常去的那家店叫 v1.9.7。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.398001536073732,
"need_memory": true
},
{
"id": "rv2-near_miss-000020",
"attribute": "办公城市",
"answerable": true,
"expected": "v1.9.7",
"reply": "您的办公城市是 v1.9.7。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 4.011829730409166,
"need_memory": true
},
{
"id": "rv2-near_miss-000188",
"attribute": "手机尾号",
"answerable": true,
"expected": "256GB",
"reply": "您的手机尾号是 256GB。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 3.3666837246021037,
"need_memory": true
},
{
"id": "rv2-near_miss-000148",
"attribute": "手机尾号",
"answerable": true,
"expected": "10.20.9.1:6379",
"reply": "您的手机尾号是 10.20.9.1:6379，您的座机尾号是 85%。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 2.9457907146042324,
"need_memory": true
},
{
"id": "rv2-multi_hop-000155",
"attribute": "",
"answerable": true,
"expected": "值班人-8175",
"reply": "负责该项目的同事对应的值班人是 值班人 -8175，编号 REF-7ZGPFQ。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000089",
"attribute": "",
"answerable": true,
"expected": "v2.8.0",
"reply": "我每天早上灌下去的那杯是偏爱 v2.8.0 口味。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.37201127265814,
"need_memory": true
},
{
"id": "rv2-noise_context-000021",
"attribute": "",
"answerable": true,
"expected": "赵工 137****8890",
"reply": "您的保险到期是 赵工 137****8890。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 4.083034336516236,
"need_memory": true
},
{
"id": "rv2-noise_context-000117",
"attribute": "",
"answerable": true,
"expected": "李工 139****3344",
"reply": "您的协议编号是 李工 139****3344。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.51884445903611,
"need_memory": true
},
{
"id": "rv2-multi_entity-000024",
"attribute": "仓库库位",
"answerable": true,
"expected": "王工 138****6621",
"reply": "项目 Alpha 的仓库库位是 王工 138****6621。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.362712320904523,
"need_memory": true
},
{
"id": "rv2-multi_hop-000043",
"attribute": "",
"answerable": true,
"expected": "值班人-2613",
"reply": "负责该项目的同事对应的值班人是 值班人 -2613。编号 REF-SE7JMG 对应的值班人是 值班人 -2613。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-multi_entity-000120",
"attribute": "值班电话",
"answerable": true,
"expected": "王工 138****6621",
"reply": "Gamma 项目有急事按王工 138****6621 找人。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 3,
"top_score": 8.3749005960278,
"need_memory": true
},
{
"id": "rv2-noise_context-000029",
"attribute": "",
"answerable": true,
"expected": "63.2%",
"reply": "您的供应商是 user，供应商，63.2%。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 2.9264157856702675,
"need_memory": true
},
{
"id": "rv2-noise_context-000037",
"attribute": "",
"answerable": true,
"expected": "B-3310",
"reply": "您的合同编号是 B-3310。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 8.527408063103344,
"need_memory": true
},
{
"id": "rv2-multi_entity-000080",
"attribute": "值班表",
"answerable": true,
"expected": "32GB",
"reply": "Beta 这周按 32GB 表轮。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.362931515002796,
"need_memory": true
},
{
"id": "rv2-multi_entity-000072",
"attribute": "周会时间",
"answerable": true,
"expected": "10.30.1.9:5432",
"reply": "张三团队的周会时间是 10.30.1.9:5432。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 3,
"top_score": 8.375409863380678,
"need_memory": true
},
{
"id": "rv2-near_miss-000116",
"attribute": "办公城市",
"answerable": true,
"expected": "每周一 09:30",
"reply": "您的办公城市是 user；办公城市；每周一 09:30。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 3.946257933775202,
"need_memory": true
},
{
"id": "rv2-multi_entity-000176",
"attribute": "周会时间",
"answerable": true,
"expected": "分机 8821",
"reply": "项目 Gamma 的周会时间是分机 8821。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 3,
"top_score": 8.369149109544557,
"need_memory": true
},
{
"id": "rv2-noise_context-000077",
"attribute": "",
"answerable": true,
"expected": "32GB",
"reply": "您的货码在 32GB。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 4.191832456142822,
"need_memory": true
},
{
"id": "rv2-multi_entity-000192",
"attribute": "保险到期",
"answerable": true,
"expected": "E-4408",
"reply": "项目 Alpha 的保险到期是 E-4408。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.367902871850985,
"need_memory": true
},
{
"id": "rv2-noise_context-000197",
"attribute": "",
"answerable": true,
"expected": "赵工 137****8890",
"reply": "您的仓库库位是 赵工 137****8890。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 4.191667331921303,
"need_memory": true
},
{
"id": "rv2-update_conflict-000170",
"attribute": "仓库库位",
"answerable": true,
"expected": "78%",
"reply": "您的货码位于 夜间 02:00。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 4.191625310646736,
"need_memory": true
},
{
"id": "rv2-near_miss-000108",
"attribute": "手机尾号",
"answerable": true,
"expected": "E-4408",
"reply": "您的设备末几位数字是 E-4408。user；手机尾号；E-4408",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 3.7148889398085996,
"need_memory": true
},
{
"id": "rv2-noise_context-000045",
"attribute": "",
"answerable": true,
"expected": "每周三 22:00",
"reply": "您的值班表是 每周三 22:00。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 4.255342933063096,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000167",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "关于您物品存放位置的信息，当前长期记忆中未包含相关记录，无法回答。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 2.852801157070653,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000073",
"attribute": "",
"answerable": true,
"expected": "10.20.3.7:5432",
"reply": "我包里常备的那本册子是 10.20.3.7:5432。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 3.7401196060261275,
"need_memory": true
},
{
"id": "rv2-long_fact-000030",
"attribute": "",
"answerable": true,
"expected": "v4.0.0-rc1",
"reply": "关于供应商这件事，会上也提过一嘴，但那次的说法是63.2%，后来没有正式确认过。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 3.2583229486472853,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000121",
"attribute": "",
"answerable": true,
"expected": "C1-105",
"reply": "我每天睡醒最早看的那个数字是 C1-105。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.51884445903611,
"need_memory": true
},
{
"id": "rv2-update_conflict-000178",
"attribute": "供应商",
"answerable": true,
"expected": "v3.14.2",
"reply": "我的供应商是 B-3310。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 2.6565719689130654,
"need_memory": true
},
{
"id": "rv2-noise_context-000125",
"attribute": "",
"answerable": true,
"expected": "10.30.7.3:6379",
"reply": "关于您上手所需的时间，当前长期记忆中未包含此信息，无法回答。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 8.539328325310356,
"need_memory": true
},
{
"id": "rv2-multi_entity-000128",
"attribute": "周会时间",
"answerable": true,
"expected": "A-7719",
"reply": "您的周会安排时间取决于具体是哪位参会者：王五固定在 10.20.9.1:6379 开周会，张三固定在 10.20.3.7:",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 8.36960156400585,
"need_memory": true
},
{
"id": "rv2-multi_entity-000064",
"attribute": "告警阈值",
"answerable": true,
"expected": "85%",
"reply": "Beta 客户触发报警的阈值是 85%。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 3,
"top_score": 8.368057877962865,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000097",
"attribute": "",
"answerable": true,
"expected": "63.2%",
"reply": "我手上那台机器是 63.2%。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 4.189765488359228,
"need_memory": true
},
{
"id": "rv2-noise_context-000085",
"attribute": "",
"answerable": true,
"expected": "v2.8.0",
"reply": "您的周会时间是 v2.8.0。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 4.246567137176742,
"need_memory": true
},
{
"id": "rv2-long_fact-000190",
"attribute": "",
"answerable": true,
"expected": "赵工 137****8890",
"reply": "关于培训周期这件事，会上也提过一嘴，但那次的说法是64GB，后来没有正式确认过。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 8.38689323936863,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000105",
"attribute": "",
"answerable": true,
"expected": "v1.9.7",
"reply": "我每天早上灌下去的那杯是偏爱 v1.9.7 口味。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.37201127265814,
"need_memory": true
},
{
"id": "rv2-update_conflict-000154",
"attribute": "培训周期",
"answerable": true,
"expected": "v1.9.7",
"reply": "我训 v1.9.7 那么久。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.376666930045054,
"need_memory": true
},
{
"id": "rv2-noise_context-000149",
"attribute": "",
"answerable": true,
"expected": "v3.14.2",
"reply": "您的值班表是 v3.14.2。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 2.615879150065484,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000063",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "您的保障结束时间未知。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 2.8673973283018523,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000153",
"attribute": "",
"answerable": true,
"expected": "v2.8.0",
"reply": "我包里常备的那本册子是 v2.8.0。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 3.9895609943661867,
"need_memory": true
},
{
"id": "rv2-long_fact-000014",
"attribute": "",
"answerable": true,
"expected": "85%",
"reply": "关于值班电话这件事，会上也提过一嘴，但那次的说法是78%，后来没有正式确认过。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 3.355738550110153,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000183",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "您的报警触发阈值未记录在长期记忆中，无法回答。user；报警触发阈值；未知",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 4.069000441972488,
"need_memory": true
},
{
"id": "rv2-long_fact-000062",
"attribute": "",
"answerable": true,
"expected": "每周三 22:00",
"reply": "关于合同编号这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认每周三 22:00，后续如果还有调整会在群里同步，暂时先按这个执行。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 8.380517067203918,
"need_memory": true
},
{
"id": "rv2-update_conflict-000018",
"attribute": "保险到期",
"answerable": true,
"expected": "v4.0.0-rc1",
"reply": "您的保险到期是 85%。user；保险到期；85%",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 4.079498370061953,
"need_memory": true
},
{
"id": "rv2-multi_hop-000027",
"attribute": "",
"answerable": true,
"expected": "值班人-5266",
"reply": "负责该项目的同事对应的值班人是 值班人 -5266。编号 REF-M37YRD 对应的值班人是 值班人 -5266。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000151",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "您的培训周期是 分机 6612。",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 2.935983789993779,
"need_memory": true
},
{
"id": "rv2-multi_entity-000000",
"attribute": "值班表",
"answerable": true,
"expected": "每周一 09:30",
"reply": "李工排班依据是每周一 09:30。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 3,
"top_score": 8.377675465173128,
"need_memory": true
},
{
"id": "rv2-near_miss-000164",
"attribute": "手机尾号",
"answerable": true,
"expected": "v1.9.7",
"reply": "您的手机尾号是 v1.9.7，座机尾号是 D-9021，按 78% 排班，货从 256GB 来。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 2.9599876489006784,
"need_memory": true
},
{
"id": "rv2-update_conflict-000194",
"attribute": "代码仓库",
"answerable": true,
"expected": "王工 138****6621",
"reply": "我源码放在 王工 138****6621。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.37575131013123,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000071",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "我排班依据是：用户；排班依据；未知。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 4.253734642020881,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000161",
"attribute": "",
"answerable": true,
"expected": "每周三 22:00",
"reply": "我手上那台机器是 每周三 22:00。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 3.497972559465167,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000127",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "您的保障结束时间是 10.20.9.1:6379。user；合同编号；10.20.9.1:6379",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 2.6956087112148968,
"need_memory": true
},
{
"id": "rv2-near_miss-000004",
"attribute": "手机尾号",
"answerable": true,
"expected": "分机 6612",
"reply": "您的手机尾号是分机 6612。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 2.962547868115764,
"need_memory": true
},
{
"id": "rv2-near_miss-000172",
"attribute": "手机尾号",
"answerable": true,
"expected": "63.2%",
"reply": "您的手机尾号是 63.2%。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 2.9593247469495347,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000017",
"attribute": "",
"answerable": true,
"expected": "C1-105",
"reply": "我周末常去的那家店叫 C1-105。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.398196269271187,
"need_memory": true
},
{
"id": "rv2-near_miss-000140",
"attribute": "手机尾号",
"answerable": true,
"expected": "91.5%",
"reply": "您的手机尾号是 91.5%。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 2.9623450335095933,
"need_memory": true
},
{
"id": "rv2-multi_entity-000144",
"attribute": "仓库库位",
"answerable": true,
"expected": "分机 8821",
"reply": "李工东西放在分机 8821 格。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.367188096819278,
"need_memory": true
},
{
"id": "rv2-multi_hop-000187",
"attribute": "",
"answerable": true,
"expected": "值班人-3660",
"reply": "负责该项目的同事对应的值班人是 值班人-3660。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-long_fact-000102",
"attribute": "",
"answerable": true,
"expected": "v2.8.0",
"reply": "关于供应商这件事，会上也提过一嘴，但那次的说法是李工 139****3344，后来没有正式确认过。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 3.273835276739059,
"need_memory": true
},
{
"id": "rv2-update_conflict-000010",
"attribute": "供应商",
"answerable": true,
"expected": "赵工 137****8890",
"reply": "您的上游供应商是 B-3310。\nuser；供应商；B-3310",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 4.012984073563698,
"need_memory": true
},
{
"id": "rv2-long_fact-000158",
"attribute": "",
"answerable": true,
"expected": "91.5%",
"reply": "关于供应商这件事，会上也提过一嘴，但那次的说法是D-9021，后来没有正式确认过。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 2.6115991130676712,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000193",
"attribute": "",
"answerable": true,
"expected": "凌晨 03:30",
"reply": "我每天早上灌下去的那杯偏爱 凌晨 03:30 的口味是未知，因为长期记忆中仅记录了时间（凌晨 03:30）和饮品名称（偏爱），未包含具体的口味信息。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.371926450482675,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000009",
"attribute": "",
"answerable": true,
"expected": "凌晨 03:30",
"reply": "我常年落脚在凌晨 03:30。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 1.8765825493415145,
"need_memory": true
},
{
"id": "rv2-multi_hop-000107",
"attribute": "",
"answerable": true,
"expected": "值班人-2126",
"reply": "负责该项目的同事对应的值班人是 值班人 -2126。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-noise_context-000069",
"attribute": "",
"answerable": true,
"expected": "10.20.3.7:5432",
"reply": "您的值班表是 10.20.3.7:5432。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 2.92918959447801,
"need_memory": true
},
{
"id": "rv2-long_fact-000150",
"attribute": "",
"answerable": true,
"expected": "32GB",
"reply": "关于发布窗口，会上也提过一嘴，但那次的说法是B3-204，后来没有正式确认过。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 8.364754177663087,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000001",
"attribute": "",
"answerable": true,
"expected": "v2.8.0",
"reply": "我手上那台机器是 v2.8.0。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 4.222987996133186,
"need_memory": true
},
{
"id": "rv2-noise_context-000141",
"attribute": "",
"answerable": true,
"expected": "64GB",
"reply": "关于您一般什么时候做上线，当前长期记忆中无相关记录，无法回答。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 8.516015553315961,
"need_memory": true
},
{
"id": "rv2-multi_entity-000008",
"attribute": "保险到期",
"answerable": true,
"expected": "王工 138****6621",
"reply": "Beta 客户保障到期时间是 王工 138****6621。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.376288132304222,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000015",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "您的代码仓库是 91.5%，仓库库位是 E-4408。",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 2.919901984287675,
"need_memory": true
},
{
"id": "rv2-noise_context-000013",
"attribute": "",
"answerable": true,
"expected": "10.20.3.7:5432",
"reply": "您提交到 N-7L4AZV7Z。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 8.367098528222034,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000135",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "您的代码仓库是 每周三 22:00。",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 2.782694476444535,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000185",
"attribute": "",
"answerable": true,
"expected": "v2.8.0",
"reply": "我手上那台机器是 v2.8.0。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 4.222987996133186,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000111",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "关于源码托管位置，当前长期记忆中未包含相关信息，无法回答。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 1.6797602105140683,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000065",
"attribute": "",
"answerable": true,
"expected": "B-3310",
"reply": "我那个数字是 B-3310。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.401031036307982,
"need_memory": true
},
{
"id": "rv2-near_miss-000100",
"attribute": "办公城市",
"answerable": true,
"expected": "E-4408",
"reply": "您的办公城市是 E-4408。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 2.8267068774764317,
"need_memory": true
},
{
"id": "rv2-multi_hop-000171",
"attribute": "",
"answerable": true,
"expected": "值班人-9282",
"reply": "负责该项目的同事对应的值班人是 值班人-9282。编号 REF-PJUBBR 对应的值班人是 值班人-9282。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-update_conflict-000066",
"attribute": "发布窗口",
"answerable": true,
"expected": "128GB",
"reply": "我一般固定 128GB 做上线。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 8.515772879382533,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000081",
"attribute": "",
"answerable": true,
"expected": "10.30.7.3:6379",
"reply": "我周末常去的那家店叫 10.30.7.3:6379。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.397619047619047,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000041",
"attribute": "",
"answerable": true,
"expected": "C1-105",
"reply": "我周末常去的那家店叫 C1-105。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.398196269271187,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000033",
"attribute": "",
"answerable": true,
"expected": "v2.8.0",
"reply": "我那个数字是 v2.8.0。user；我每天睡醒最早看的那个数字是 v2.8.0。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.519136638615494,
"need_memory": true
},
{
"id": "rv2-multi_hop-000083",
"attribute": "",
"answerable": true,
"expected": "值班人-7311",
"reply": "负责该项目的同事对应的值班人是 值班人-7311。编号 REF-LQPXN5 对应的值班人是 值班人-7311。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-near_miss-000028",
"attribute": "办公城市",
"answerable": true,
"expected": "256GB",
"reply": "您的办公城市是 256GB。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 2.8856920200014633,
"need_memory": true
},
{
"id": "rv2-long_fact-000094",
"attribute": "",
"answerable": true,
"expected": "256GB",
"reply": "关于发布窗口，之前散会时我们临时定了下来，当时讨论得比较久，最后确认256GB，后续如果还有调整会在群里同步，暂时先按这个执行。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 8.364802951937156,
"need_memory": true
},
{
"id": "rv2-update_conflict-000002",
"attribute": "代码仓库",
"answerable": true,
"expected": "每周一 09:30",
"reply": "您的代码提交到 D-9021。user；代码仓库；D-9021",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 2.8594369254356775,
"need_memory": true
},
{
"id": "rv2-multi_entity-000136",
"attribute": "值班电话",
"answerable": true,
"expected": "E-4408",
"reply": "Delta 的值班电话是 E-4408。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 3,
"top_score": 8.36674541052584,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000087",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "关于您这周按什么表轮的问题，当前长期记忆中未包含该信息。已知信息如下：user；主库地址；每周三 22:00；合同编号；凌晨 03:30；",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 2.934849732169521,
"need_memory": true
},
{
"id": "rv2-update_conflict-000098",
"attribute": "告警阈值",
"answerable": true,
"expected": "10.30.1.9:5432",
"reply": "我超过 10.30.1.9:5432 就报警。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 4.1142092482916945,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000199",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "您的提交位置是 每周一 09:30。user；仓库库位；每周一 09:30",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 4.059683012089394,
"need_memory": true
},
{
"id": "rv2-near_miss-000076",
"attribute": "办公城市",
"answerable": true,
"expected": "钱工 135****2211",
"reply": "您的办公城市是 钱工 135****2211。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 2.914480213137743,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000137",
"attribute": "",
"answerable": true,
"expected": "10.20.9.1:6379",
"reply": "我包里常备的那本册子是 10.20.9.1:6379。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 3.7026806769451643,
"need_memory": true
},
{
"id": "rv2-near_miss-000052",
"attribute": "手机尾号",
"answerable": true,
"expected": "91.5%",
"reply": "您的手机尾号是 91.5%，座机尾号是 78%。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 2.9623450335095933,
"need_memory": true
},
{
"id": "rv2-multi_entity-000096",
"attribute": "主库地址",
"answerable": true,
"expected": "10.20.9.1:6379",
"reply": "王五的写库连接地址是 10.20.9.1:6379。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.368147521943593,
"need_memory": true
},
{
"id": "rv2-multi_hop-000051",
"attribute": "",
"answerable": true,
"expected": "值班人-3738",
"reply": "负责该项目的同事对应的值班人是 值班人-3738。编号 REF-S63RSN 对应的值班人是 值班人-3738。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-multi_hop-000075",
"attribute": "",
"answerable": true,
"expected": "值班人-6399",
"reply": "负责该项目的同事对应的值班人是 值班人-6399。编号 REF-QPMDFP 对应的值班人是 值班人-6399。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-update_conflict-000034",
"attribute": "代码仓库",
"answerable": true,
"expected": "32GB",
"reply": "我提交到 D-9021。user；代码仓库；D-9021",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 2.8186671648489976,
"need_memory": true
},
{
"id": "rv2-long_fact-000198",
"attribute": "",
"answerable": true,
"expected": "v1.9.7",
"reply": "关于周会时间，会上也提过一嘴，但那次的说法是每周一 09:30，后来没有正式确认过。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 8.364705882352942,
"need_memory": true
},
{
"id": "rv2-multi_entity-000088",
"attribute": "值班电话",
"answerable": true,
"expected": "85%",
"reply": "Delta 客户有急事按 85% 这个号找人。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 3,
"top_score": 8.3752178138612,
"need_memory": true
},
{
"id": "rv2-multi_hop-000011",
"attribute": "",
"answerable": true,
"expected": "值班人-6542",
"reply": "负责该项目的同事对应的值班人是 值班人-6542。编号 REF-NVN3HY 对应的值班人是 值班人-6542。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-update_conflict-000082",
"attribute": "发布窗口",
"answerable": true,
"expected": "每周三 22:00",
"reply": "我固定每周三 22:00 做上线。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 8.515832512409528,
"need_memory": true
},
{
"id": "rv2-multi_hop-000019",
"attribute": "",
"answerable": true,
"expected": "值班人-3411",
"reply": "负责该项目的同事对应的值班人是 值班人-3411。编号 REF-TWTM4V 对应的值班人是 值班人-3411。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000103",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "您的培训周期是 每周三 22:00。",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 2.9904550989399743,
"need_memory": true
},
{
"id": "rv2-multi_entity-000112",
"attribute": "周会时间",
"answerable": true,
"expected": "10.20.3.7:5432",
"reply": "张三团队的周会时间是 10.20.3.7:5432。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 3,
"top_score": 8.375409863380678,
"need_memory": true
},
{
"id": "rv2-multi_hop-000059",
"attribute": "",
"answerable": true,
"expected": "值班人-7047",
"reply": "负责该项目的同事对应的值班人是 值班人 -7047。编号 REF-F99DWT 对应的值班人是 值班人 -7047。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-noise_context-000093",
"attribute": "",
"answerable": true,
"expected": "D-9021",
"reply": "您的主库地址是 D-9021。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 8.52658972686322,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000031",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "关于您 weekly 的会安排时间，当前长期记忆中未包含相关信息，无法回答。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 2.9848565223567585,
"need_memory": true
},
{
"id": "rv2-update_conflict-000090",
"attribute": "发布窗口",
"answerable": true,
"expected": "10.30.1.9:5432",
"reply": "我一般固定 10.30.1.9:5432 做上线。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 8.515953836117932,
"need_memory": true
},
{
"id": "rv2-noise_context-000053",
"attribute": "",
"answerable": true,
"expected": "D-9021",
"reply": "抱歉，我不知道您具体指的是哪个设备或场景的报警阈值，因为缺乏相关证据。请提供具体的设备名称或上下文，以便我准确回答。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 3.923231945601471,
"need_memory": true
},
{
"id": "rv2-update_conflict-000138",
"attribute": "仓库库位",
"answerable": true,
"expected": "夜间 02:00",
"reply": "您的东西放在 63.2% 的仓库库位。user 的仓库库位是 63.2%。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 4.11762958432377,
"need_memory": true
},
{
"id": "rv2-long_fact-000142",
"attribute": "",
"answerable": true,
"expected": "A-7719",
"reply": "周会时间已确认为 A-7719，后续如有调整会在群里同步，暂时先按这个执行。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.365665849742912,
"need_memory": true
},
{
"id": "rv2-noise_context-000173",
"attribute": "",
"answerable": true,
"expected": "32GB",
"reply": "您提交到 N-QG46HPKB。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 8.367098528222034,
"need_memory": true
},
{
"id": "rv2-long_fact-000078",
"attribute": "",
"answerable": true,
"expected": "A-7719",
"reply": "关于培训周期，之前散会时我们临时定了下来，当时讨论得比较久，最后确认A-7719，后续如果还有调整会在群里同步，暂时先按这个执行。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.370457927209625,
"need_memory": true
},
{
"id": "rv2-update_conflict-000026",
"attribute": "供应商",
"answerable": true,
"expected": "78%",
"reply": "您的货是由 v3.14.2 供的。\nuser；供应商；v3.14.2",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 2.793988853564169,
"need_memory": true
},
{
"id": "rv2-long_fact-000182",
"attribute": "",
"answerable": true,
"expected": "分机 8821",
"reply": "关于告警阈值这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认分机 8821，后续如果还有调整会在群里同步，暂时先按这个执行。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 4.021412185140287,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000177",
"attribute": "",
"answerable": true,
"expected": "10.30.1.9:5432",
"reply": "我那个数字是 10.30.1.9:5432。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.518988863132819,
"need_memory": true
},
{
"id": "rv2-update_conflict-000042",
"attribute": "培训周期",
"answerable": true,
"expected": "256GB",
"reply": "我训 256GB 那么久。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 8.53993810315142,
"need_memory": true
},
{
"id": "rv2-long_fact-000110",
"attribute": "",
"answerable": true,
"expected": "夜间 02:00",
"reply": "关于值班电话这件事，会上也提过一嘴，但那次的说法是 10.30.1.9:5432，后来没有正式确认过。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 3.9931269983591773,
"need_memory": true
},
{
"id": "rv2-long_fact-000126",
"attribute": "",
"answerable": true,
"expected": "分机 8821",
"reply": "关于告警阈值这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认分机 8821，后续如果还有调整会在群里同步，暂时先按这个执行。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 4.021412185140287,
"need_memory": true
},
{
"id": "rv2-near_miss-000156",
"attribute": "办公城市",
"answerable": true,
"expected": "A-7719",
"reply": "您的办公城市是 A-7719。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 3.982735022703425,
"need_memory": true
},
{
"id": "rv2-near_miss-000180",
"attribute": "办公城市",
"answerable": true,
"expected": "32GB",
"reply": "您的办公城市是 32GB。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 2.885920842232756,
"need_memory": true
},
{
"id": "rv2-multi_hop-000139",
"attribute": "",
"answerable": true,
"expected": "值班人-5479",
"reply": "负责该项目的同事对应的值班人是 值班人 -5479。编号 REF-CMJ4GB 对应的值班人是 值班人 -5479。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000119",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "您的上线时间是 v3.14.2。user；周会时间；v3.14.2",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.516015553315961,
"need_memory": true
},
{
"id": "rv2-multi_hop-000179",
"attribute": "",
"answerable": true,
"expected": "值班人-2662",
"reply": "负责该项目的同事对应的值班人是 值班人 -2662。编号 REF-KEZE8S 对应的值班人是 值班人 -2662。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-long_fact-000054",
"attribute": "",
"answerable": true,
"expected": "A-7719",
"reply": "关于合同编号这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认A-7719，后续如果还有调整会在群里同步，暂时先按这个执行。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 8.380517067203918,
"need_memory": true
},
{
"id": "rv2-multi_entity-000168",
"attribute": "告警阈值",
"answerable": true,
"expected": "v2.8.0",
"reply": "张三的告警阈值是 v2.8.0。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 3,
"top_score": 8.367568209223156,
"need_memory": true
},
{
"id": "rv2-update_conflict-000050",
"attribute": "告警阈值",
"answerable": true,
"expected": "v1.9.7",
"reply": "您的告警阈值是 10.30.7.3:6379。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 3.7124229212736184,
"need_memory": true
},
{
"id": "rv2-multi_entity-000016",
"attribute": "供应商",
"answerable": true,
"expected": "B-3310",
"reply": "李工的供应商是 B-3310。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 8.375641025641025,
"need_memory": true
},
{
"id": "rv2-long_fact-000086",
"attribute": "",
"answerable": true,
"expected": "256GB",
"reply": "关于发布窗时段变更，之前散会时我们临时定了下来，当时讨论得比较久，最后确认256GB，后续如果还有调整会在群里同步，暂时先按这个执行。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 8.366927823166233,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000007",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "您的培训周期是 10.30.1.9:5432。",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 2.931934316913914,
"need_memory": true
},
{
"id": "rv2-noise_context-000133",
"attribute": "",
"answerable": true,
"expected": "分机 6612",
"reply": "您的告警阈值是 分机 6612。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 4.066645118859324,
"need_memory": true
},
{
"id": "rv2-noise_context-000061",
"attribute": "",
"answerable": true,
"expected": "v4.0.0-rc1",
"reply": "您的协议编号是 v4.0.0-rc1。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.51884445903611,
"need_memory": true
},
{
"id": "rv2-multi_hop-000091",
"attribute": "",
"answerable": true,
"expected": "值班人-5068",
"reply": "负责该项目的同事对应的值班人是 值班人-5068。编号 REF-WUV4EX 对应的值班人是 值班人-5068。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-update_conflict-000074",
"attribute": "合同编号",
"answerable": true,
"expected": "64GB",
"reply": "我签的是 64GB 号合同。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 8.527408063103344,
"need_memory": true
},
{
"id": "rv2-update_conflict-000130",
"attribute": "告警阈值",
"answerable": true,
"expected": "B-3310",
"reply": "您的告警阈值是每周三 22:00。user；告警阈值；每周三 22:00",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 3.8470214095228976,
"need_memory": true
},
{
"id": "rv2-noise_context-000189",
"attribute": "",
"answerable": true,
"expected": "夜间 02:00",
"reply": "您的合同编号是 夜间 02:00。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 8.527199633743077,
"need_memory": true
},
{
"id": "rv2-noise_context-000157",
"attribute": "",
"answerable": true,
"expected": "D-9021",
"reply": "您的团队碰会时间是 D-9021。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 8.516015553315961,
"need_memory": true
},
{
"id": "rv2-multi_hop-000123",
"attribute": "",
"answerable": true,
"expected": "值班人-7755",
"reply": "负责该项目的同事对应的值班人是 值班人 -7755。编号 REF-XQ23AR 对应的值班人是 值班人 -7755。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-multi_entity-000040",
"attribute": "保险到期",
"answerable": true,
"expected": "D-9021",
"reply": "客户 Beta 的保险到期是 D-9021。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.376511865123991,
"need_memory": true
},
{
"id": "rv2-multi_entity-000152",
"attribute": "仓库库位",
"answerable": true,
"expected": "B3-204",
"reply": "Delta 客户东西放在 B3-204 格。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 8.372616563651595,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000049",
"attribute": "",
"answerable": true,
"expected": "每周一 09:30",
"reply": "我周末常去的那家店叫 每周一 09:30。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.397809144373376,
"need_memory": true
},
{
"id": "rv2-near_miss-000036",
"attribute": "办公城市",
"answerable": true,
"expected": "D-9021",
"reply": "您的办公城市是 D-9021。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 2.9146409106630458,
"need_memory": true
},
{
"id": "rv2-multi_entity-000184",
"attribute": "培训周期",
"answerable": true,
"expected": "64GB",
"reply": "客户 Delta 的培训周期是 64GB。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 3,
"top_score": 8.37654971220268,
"need_memory": true
},
{
"id": "rv2-multi_hop-000131",
"attribute": "",
"answerable": true,
"expected": "值班人-5200",
"reply": "负责该项目的同事对应的值班人是 值班人-5200。编号 REF-VPML2S 对应的值班人是 值班人-5200。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000079",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "您的协议编号是：user；协议编号；不知道。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 1.6796213912963864,
"need_memory": true
},
{
"id": "rv2-multi_entity-000032",
"attribute": "保险到期",
"answerable": true,
"expected": "王工 138****6621",
"reply": "Gamma 保障的保险到期时间是 王工 138****6621。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.367751790062417,
"need_memory": true
},
{
"id": "rv2-multi_hop-000003",
"attribute": "",
"answerable": true,
"expected": "值班人-5259",
"reply": "负责该项目的同事对应的值班人是 值班人-5259。编号 REF-9X6UYE 对应的值班人是 值班人-5259。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000039",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "您的报警触发阈值未在当前长期记忆中记录，无法提供具体数值。已知信息包括：user；发布窗口；凌晨 03:30；user；值班电话；v1.9.7；user；培训周期",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 3.825621356089628,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000057",
"attribute": "",
"answerable": true,
"expected": "256GB",
"reply": "我常年落脚在 256GB。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 1.8764250194619514,
"need_memory": true
},
{
"id": "rv2-noise_context-000005",
"attribute": "",
"answerable": true,
"expected": "85%",
"reply": "您的仓库库位是 85%。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 2.925214541927536,
"need_memory": true
},
{
"id": "rv2-long_fact-000022",
"attribute": "",
"answerable": true,
"expected": "E-4408",
"reply": "关于发布窗时段变更，之前散会时临时定下来的是 E-4408，后续如果还有调整会在群里同步，暂时先按这个执行。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 8.366872047649968,
"need_memory": true
},
{
"id": "rv2-multi_entity-000056",
"attribute": "值班电话",
"answerable": true,
"expected": "256GB",
"reply": "李工轮班时该拨打的号码是 256GB。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 3,
"top_score": 8.375202432454547,
"need_memory": true
},
{
"id": "rv2-long_fact-000118",
"attribute": "",
"answerable": true,
"expected": "王工 138****6621",
"reply": "关于周会时间，之前散会时我们临时定了下来，当时讨论得比较久，最后确认王工 138****6621，后续如果还有调整会在群里同步，暂时先按这个执行。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 8.36465805821864,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000095",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "您的轮表信息未知。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 1.6675797629356381,
"need_memory": true
},
{
"id": "rv2-update_conflict-000058",
"attribute": "主库地址",
"answerable": true,
"expected": "钱工 135****2211",
"reply": "我主库落在 钱工 135****2211。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 8.526288132304222,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000191",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "您的值班表是 B3-204，供应商是 赵工 137****8890。",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 4.234734808299918,
"need_memory": true
},
{
"id": "rv2-multi_entity-000160",
"attribute": "保险到期",
"answerable": true,
"expected": "v1.9.7",
"reply": "客户 Beta 的保险到期是 v1.9.7。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.362766073029704,
"need_memory": true
},
{
"id": "rv2-update_conflict-000146",
"attribute": "保险到期",
"answerable": true,
"expected": "每周三 22:00",
"reply": "您的保险失效时间是 李工 139****3344。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 4.117351045888133,
"need_memory": true
},
{
"id": "rv2-near_miss-000124",
"attribute": "手机尾号",
"answerable": true,
"expected": "李工 139****3344",
"reply": "您的手机尾号是 李工 139****3344。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 2.967079817975403,
"need_memory": true
},
{
"id": "rv2-near_miss-000012",
"attribute": "手机尾号",
"answerable": true,
"expected": "10.20.3.7:5432",
"reply": "您的手机尾号是 10.20.3.7:5432。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 2.8124214405308816,
"need_memory": true
},
{
"id": "rv2-near_miss-000060",
"attribute": "办公城市",
"answerable": true,
"expected": "78%",
"reply": "您的办公城市是 78%。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 2.9146409106630458,
"need_memory": true
},
{
"id": "rv2-update_conflict-000122",
"attribute": "告警阈值",
"answerable": true,
"expected": "B-3310",
"reply": "我超过 B-3310 就报警。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.368298126367785,
"need_memory": true
},
{
"id": "rv2-multi_hop-000163",
"attribute": "",
"answerable": true,
"expected": "值班人-1122",
"reply": "负责该项目的同事对应的值班人是 值班人-1122。编号 REF-4CPD4V 对应的值班人是 值班人-1122。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-noise_context-000109",
"attribute": "",
"answerable": true,
"expected": "B-3310",
"reply": "您的值班表是 B-3310。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 2.577218743873267,
"need_memory": true
},
{
"id": "rv2-update_conflict-000114",
"attribute": "仓库库位",
"answerable": true,
"expected": "v3.14.2",
"reply": "您的东西放在每周三 22:00。实体：user；仓库库位：每周三 22:00。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 3.9810770153876582,
"need_memory": true
},
{
"id": "rv2-update_conflict-000162",
"attribute": "告警阈值",
"answerable": true,
"expected": "64GB",
"reply": "我超过 64GB 就报警。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.368380365552344,
"need_memory": true
},
{
"id": "rv2-multi_entity-000048",
"attribute": "周会时间",
"answerable": true,
"expected": "256GB",
"reply": "Beta 客户 weekly 的会安排在 256GB。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 3,
"top_score": 8.379966456342837,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000023",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "关于您的 weekly 会议安排，当前长期记忆中未包含相关信息，无法回答。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 2.6294802729640434,
"need_memory": true
},
{
"id": "rv2-multi_hop-000195",
"attribute": "",
"answerable": true,
"expected": "值班人-8311",
"reply": "负责该项目的同事对应的值班人是 值班人-8311。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000175",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "关于您的排班依据，当前长期记忆中未包含相关信息，无法回答。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 4.250994098107883,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000025",
"attribute": "",
"answerable": true,
"expected": "A-7719",
"reply": "我手上那台机器是 A-7719。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 4.044519101831213,
"need_memory": true
},
{
"id": "rv2-multi_hop-000035",
"attribute": "",
"answerable": true,
"expected": "值班人-8344",
"reply": "负责该项目的同事对应的值班人是 值班人-8344。编号 REF-QGBYWN 对应的值班人是 值班人-8344。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000159",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "我的数据落在 10.20.3.7:5432。user；代码仓库；10.20.3.7:5432",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.516015553315961,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000143",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "关于上手所需时间，当前长期记忆中未包含相关信息，无法回答。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.54025382429497,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000047",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "您的轮班拨号号码是 王工 138****6621。",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 3.7929448088215723,
"need_memory": true
},
{
"id": "rv2-near_miss-000092",
"attribute": "手机尾号",
"answerable": true,
"expected": "v4.0.0-rc1",
"reply": "您的随身设备末几位数字是 user；手机尾号；v4.0.0-rc1。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 3.5110823483997535,
"need_memory": true
},
{
"id": "rv2-long_fact-000166",
"attribute": "",
"answerable": true,
"expected": "李工 139****3344",
"reply": "关于值班表这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认李工 139****3344，后续如果还有调整会在群里同步，暂时先按这个执行。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 8.3663663417677,
"need_memory": true
},
{
"id": "rv2-long_fact-000046",
"attribute": "",
"answerable": true,
"expected": "A-7719",
"reply": "关于值班表，之前散会时临时定下来按 A-7719 轮，后续如有调整会在群里同步。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 2,
"top_score": 8.36670797197578,
"need_memory": true
},
{
"id": "rv2-long_fact-000174",
"attribute": "",
"answerable": true,
"expected": "v3.14.2",
"reply": "关于主库地址，之前散会时临时定下来的是 v3.14.2，后续若有调整会在群里同步，暂时先按这个执行。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 8.376119564612543,
"need_memory": true
},
{
"id": "rv2-near_miss-000132",
"attribute": "手机尾号",
"answerable": true,
"expected": "v4.0.0-rc1",
"reply": "您的座机尾号是 v2.8.0，手机尾号是 v4.0.0-rc1。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 2.940968700822523,
"need_memory": true
},
{
"id": "rv2-noise_context-000165",
"attribute": "",
"answerable": true,
"expected": "10.30.1.9:5432",
"reply": "您的仓库库位是 10.30.1.9:5432。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 2.7442423331988484,
"need_memory": true
},
{
"id": "rv2-noise_context-000101",
"attribute": "",
"answerable": true,
"expected": "王工 138****6621",
"reply": "您的发布窗口是 王工 138****6621。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 9,
"records_active": 9,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 8.515832512409528,
"need_memory": true
},
{
"id": "rv2-unknown_attribute-000055",
"attribute": "",
"answerable": false,
"expected": "",
"reply": "关于您这周按什么表轮，当前长期记忆中未包含该信息。已知信息如下：user；培训周期；78%；user；保险到期；v1.9.7；user；合同编号；1",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 2.9328151474153596,
"need_memory": true
},
{
"id": "rv2-near_miss-000044",
"attribute": "手机尾号",
"answerable": true,
"expected": "256GB",
"reply": "您的随身设备末几位数字是 256GB。user；手机尾号；256GB",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 3.5378855318625115,
"need_memory": true
},
{
"id": "rv2-multi_hop-000067",
"attribute": "",
"answerable": true,
"expected": "值班人-3951",
"reply": "负责该项目的同事对应的值班人是 值班人 -3951。编号 REF-BNHNGX 对应的值班人是 值班人 -3951。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-near_miss-000068",
"attribute": "手机尾号",
"answerable": true,
"expected": "v2.8.0",
"reply": "您的手机尾号是 v2.8.0。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 2.9610466954358827,
"need_memory": true
},
{
"id": "rv2-long_fact-000006",
"attribute": "",
"answerable": true,
"expected": "128GB",
"reply": "关于合同编号这件事，会上也提过一嘴，但那次的说法是63.2%，后来没有正式确认过。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 4,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 3,
"top_score": 4.12988681260015,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000129",
"attribute": "",
"answerable": true,
"expected": "B-3310",
"reply": "我每天早上灌下去的那杯偏爱 B-3310 的口味信息在现有记录中未明确说明，无法确认具体口味。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.37209708691208,
"need_memory": true
},
{
"id": "rv2-multi_hop-000115",
"attribute": "",
"answerable": true,
"expected": "值班人-9666",
"reply": "负责该项目的同事对应的值班人是 值班人-9666。编号 REF-GAXKV6 对应的值班人是 值班人-9666。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 5,
"top_score": 8.40043619793543,
"need_memory": true
},
{
"id": "rv2-multi_entity-000104",
"attribute": "合同编号",
"answerable": true,
"expected": "夜间 02:00",
"reply": "Beta 协议编号是 夜间 02:00。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 3,
"records_active": 3,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.376352313834737,
"need_memory": true
},
{
"id": "rv2-alias_paraphrase-000169",
"attribute": "",
"answerable": true,
"expected": "v4.0.0-rc1",
"reply": "我每天早上灌下去的那杯是偏爱 v4.0.0-rc1 口味。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 5,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.371926450482675,
"need_memory": true
},
{
"id": "rv2-long_fact-000038",
"attribute": "",
"answerable": true,
"expected": "李工 139****3344",
"reply": "关于培训周期这件事，会上也提过一嘴，但那次的说法是85%，后来没有正式确认过。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 8.38689323936863,
"need_memory": true
},
{
"id": "rv2-long_fact-000134",
"attribute": "",
"answerable": true,
"expected": "v2.8.0",
"reply": "关于保险到期这件事，会上也提过一嘴，但那次的说法是10.30.1.9:5432，后来没有正式确认过。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 4,
"records_active": 4,
"stop_reason": "evidence_found",
"records_selected": 4,
"top_score": 8.369964096909316,
"need_memory": true
},
{
"id": "rv2-update_conflict-000186",
"attribute": "培训周期",
"answerable": true,
"expected": "每周三 22:00",
"reply": "我训每周三 22:00 那么久。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 5,
"records_active": 2,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.376319840237885,
"need_memory": true
}
]
}
}
Reference in New Issue View Git Blame Copy Permalink