Natural Memory NM2.1: 记忆路由器分叉、数据集缺陷修复与全轴评测证据
- 引入 MemoryRouterXL 与 v5/v6 流式多线程训练/编码管线 - 修复 prepare_memory_router_dataset 候选池重建缺陷（mega 家族 3568x 加速，输出逐字节相同） - 修复 v5 被破坏的拒答与多跳标签（train 未知样本 319 -> 16319，multi_hop 平均正例 1.00 -> 2.00） - 同存储预算下 V2-128 v6 逐轴 22/22 通过：Top-1 41.12% -> 94.62%，未知拒答 0.00% -> 100.00% - 记录三条被实测推翻的显然优化（logits_to_keep=1 反而慢 55%、XL 容量未带来收益） - 记忆手术跨架构可移植性 14/14，读写关闭时与原生模型逐位相同
This commit is contained in:
WpyQwq
@@ -0,0 +1,200 @@
|
||||
{"id":"realistic-multi_entity-00000","group_id":"realistic-multi_entity-00000","source":"realistic_memory_eval","family":"multi_entity","query":"我们组那位张三固定在哪个位置办公？","candidates":[{"text":"张三常驻机房在 v2.8.0。","kind":"fact","entity":"user","attribute":""},{"text":"客户 Beta平时待的机房是 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"我平时待的机房是 78%。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["v2.8.0"],"hop_count":1,"answer":"v2.8.0","attribute":"常驻机房"}}
|
||||
{"id":"realistic-alias_paraphrase-00001","group_id":"realistic-alias_paraphrase-00001","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我早上那杯是什么口味？","candidates":[{"text":"我的告警阈值是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我每天早上灌下去的那杯偏爱 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 A-7719。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["B-3310"],"hop_count":1,"answer":"B-3310"}}
|
||||
{"id":"realistic-update_conflict-00002","group_id":"realistic-update_conflict-00002","source":"realistic_memory_eval","family":"update_conflict","query":"我哪条线一破就叫？","candidates":[{"text":"我的告警阈值是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 v2.8.0。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我超过 128GB 就报警。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 78%。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3],"positive_index":3,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["128GB"],"hop_count":1,"answer":"128GB","attribute":"告警阈值","superseded_value":"78%"}}
|
||||
{"id":"realistic-multi_hop-00003","group_id":"realistic-multi_hop-00003","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-VWK3DU 对应的值班人是 值班人-2332。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-9KXDQY。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-9KXDQY 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-47HJBF 对应的值班人是 值班人-3100。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-SEBPGB 对应的值班人是 值班人-1050。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1,2],"positive_index":1,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00004","group_id":"realistic-near_miss-00004","source":"realistic_memory_eval","family":"near_miss","query":"我家座机最后几位数字是啥？","candidates":[{"text":"我的常住城市是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的办公城市是 B-3310。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["分机 8821"],"hop_count":1,"answer":"分机 8821","attribute":"常住城市"}}
|
||||
{"id":"realistic-noise_context-00005","group_id":"realistic-noise_context-00005","source":"realistic_memory_eval","family":"noise_context","query":"我现在对接的是哪一版？","candidates":[{"text":"这是普通对话噪声：验证用户-610771 暂时提到一个无关编号 N-AUZ5HPPJ，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-832734 暂时提到一个无关编号 N-BTV6W9E2，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-146386 暂时提到一个无关编号 N-E4LZBS9K，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 王工 138****6621。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-637927 暂时提到一个无关编号 N-9L765TLB，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-010982 暂时提到一个无关编号 N-GD62NDK6，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-342684 暂时提到一个无关编号 N-Z3KBMBZ5，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-309172 暂时提到一个无关编号 N-QUZ2ZCPQ，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-760933 暂时提到一个无关编号 N-75LZQM7K，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3],"positive_index":3,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["王工 138****6621"],"hop_count":1,"answer":"王工 138****6621"}}
|
||||
{"id":"realistic-long_fact-00006","group_id":"realistic-long_fact-00006","source":"realistic_memory_eval","family":"long_fact","query":"我缓冲配了多少？","candidates":[{"text":"关于缓存容量这件事，会上也提过一嘴，但那次的说法是128GB，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于缓存容量这件事，会上也提过一嘴，但那次的说法是夜间 02:00，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于缓存容量这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认A-7719，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""},{"text":"关于缓存容量这件事，会上也提过一嘴，但那次的说法是夜间 02:00，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["A-7719"],"hop_count":1,"answer":"A-7719"}}
|
||||
{"id":"realistic-unknown_attribute-00007","group_id":"realistic-unknown_attribute-00007","source":"realistic_memory_eval","family":"unknown_attribute","query":"我有急事按哪个号找人？","candidates":[{"text":"我的应急联系人是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 王工 138****6621。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 李工 139****3344。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00008","group_id":"realistic-multi_entity-00008","source":"realistic_memory_eval","family":"multi_entity","query":"Beta 客户固定在哪个位置办公？","candidates":[{"text":"李工平时待的机房是 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"客户 Beta常驻机房在 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"张三平时待的机房是 A-7719。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["夜间 02:00"],"hop_count":1,"answer":"夜间 02:00","attribute":"常驻机房"}}
|
||||
{"id":"realistic-alias_paraphrase-00009","group_id":"realistic-alias_paraphrase-00009","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我那个数字是多少？","candidates":[{"text":"我每天睡醒最早看的那个数字是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 每周三 22:00。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["B-3310"],"hop_count":1,"answer":"B-3310"}}
|
||||
{"id":"realistic-update_conflict-00010","group_id":"realistic-update_conflict-00010","source":"realistic_memory_eval","family":"update_conflict","query":"我缓冲配了多少？","candidates":[{"text":"我的缓存容量是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我缓存开到 王工 138****6621。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[4],"positive_index":4,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["王工 138****6621"],"hop_count":1,"answer":"王工 138****6621","attribute":"缓存容量","superseded_value":"10.20.3.7:5432"}}
|
||||
{"id":"realistic-multi_hop-00011","group_id":"realistic-multi_hop-00011","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-ATRPWY 对应的值班人是 值班人-3032。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-36F3UA 对应的值班人是 值班人-6978。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-EDGJJC 对应的值班人是 值班人-1391。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-TGG9C4。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-TGG9C4 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3,4],"positive_index":3,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00012","group_id":"realistic-near_miss-00012","source":"realistic_memory_eval","family":"near_miss","query":"我家座机最后几位数字是啥？","candidates":[{"text":"我的办公城市是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我的常住城市是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 v2.8.0。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["78%"],"hop_count":1,"answer":"78%","attribute":"常住城市"}}
|
||||
{"id":"realistic-noise_context-00013","group_id":"realistic-noise_context-00013","source":"realistic_memory_eval","family":"noise_context","query":"我平时在哪间屋子干活？","candidates":[{"text":"这是普通对话噪声：验证用户-789083 暂时提到一个无关编号 N-LN72EE53，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-133567 暂时提到一个无关编号 N-85MS3HXT，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-808479 暂时提到一个无关编号 N-QQABUEHB，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-212928 暂时提到一个无关编号 N-9XRJ4UGL，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-915797 暂时提到一个无关编号 N-WHAVHKX9，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-104010 暂时提到一个无关编号 N-UQNW6H9C，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-778134 暂时提到一个无关编号 N-WNM8LNPS，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-026605 暂时提到一个无关编号 N-P4TPVADJ，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[4],"positive_index":4,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["v3.14.2"],"hop_count":1,"answer":"v3.14.2"}}
|
||||
{"id":"realistic-long_fact-00014","group_id":"realistic-long_fact-00014","source":"realistic_memory_eval","family":"long_fact","query":"我紧急情况下找谁接手？","candidates":[{"text":"关于应急联系人这件事，会上也提过一嘴，但那次的说法是B-3310，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于应急联系人这件事，会上也提过一嘴，但那次的说法是10.20.9.1:6379，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于应急联系人这件事，会上也提过一嘴，但那次的说法是78%，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于应急联系人这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认91.5%，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3],"positive_index":3,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["91.5%"],"hop_count":1,"answer":"91.5%"}}
|
||||
{"id":"realistic-unknown_attribute-00015","group_id":"realistic-unknown_attribute-00015","source":"realistic_memory_eval","family":"unknown_attribute","query":"我紧急情况下找谁接手？","candidates":[{"text":"我的发布窗口是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 A-7719。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00016","group_id":"realistic-multi_entity-00016","source":"realistic_memory_eval","family":"multi_entity","query":"Beta 客户挑哪个时段变更？","candidates":[{"text":"项目 Alpha固定 78% 做上线。","kind":"fact","entity":"user","attribute":""},{"text":"我固定 10.20.9.1:6379 做上线。","kind":"fact","entity":"user","attribute":""},{"text":"客户 Beta的发布窗口是 每周三 22:00。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["每周三 22:00"],"hop_count":1,"answer":"每周三 22:00","attribute":"发布窗口"}}
|
||||
{"id":"realistic-alias_paraphrase-00017","group_id":"realistic-alias_paraphrase-00017","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我那个数字是多少？","candidates":[{"text":"我常驻机房在 王工 138****6621。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我每天睡醒最早看的那个数字是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 64GB。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3],"positive_index":3,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["夜间 02:00"],"hop_count":1,"answer":"夜间 02:00"}}
|
||||
{"id":"realistic-update_conflict-00018","group_id":"realistic-update_conflict-00018","source":"realistic_memory_eval","family":"update_conflict","query":"我数据落在哪一个地址？","candidates":[{"text":"我的主库地址是 王工 138****6621。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我主库落在 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 v3.14.2。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["B3-204"],"hop_count":1,"answer":"B3-204","attribute":"主库地址","superseded_value":"每周三 22:00"}}
|
||||
{"id":"realistic-multi_hop-00019","group_id":"realistic-multi_hop-00019","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-2JPTMG 对应的值班人是 值班人-1605。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-8NFSFM 对应的值班人是 值班人-1149。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-2YRA4K。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-QY6AJ3 对应的值班人是 值班人-2870。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-2YRA4K 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2,4],"positive_index":2,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00020","group_id":"realistic-near_miss-00020","source":"realistic_memory_eval","family":"near_miss","query":"我家座机最后几位数字是啥？","candidates":[{"text":"我的值班电话是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的宿舍楼层是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的工位楼层是 王工 138****6621。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 B-3310。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["A-7719"],"hop_count":1,"answer":"A-7719","attribute":"宿舍楼层"}}
|
||||
{"id":"realistic-noise_context-00021","group_id":"realistic-noise_context-00021","source":"realistic_memory_eval","family":"noise_context","query":"我数据落在哪一个地址？","candidates":[{"text":"这是普通对话噪声：验证用户-567992 暂时提到一个无关编号 N-K5M73ANN，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-739223 暂时提到一个无关编号 N-PPL6LS27，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-671419 暂时提到一个无关编号 N-AKSHMKX3，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-882872 暂时提到一个无关编号 N-88PXHYN6，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-874995 暂时提到一个无关编号 N-X36GCKRG，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-536404 暂时提到一个无关编号 N-LVVTCQT2，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-923111 暂时提到一个无关编号 N-RSCG6FLT，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-284329 暂时提到一个无关编号 N-DW3RNUWQ，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[7],"positive_index":7,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["10.20.9.1:6379"],"hop_count":1,"answer":"10.20.9.1:6379"}}
|
||||
{"id":"realistic-long_fact-00022","group_id":"realistic-long_fact-00022","source":"realistic_memory_eval","family":"long_fact","query":"我联调用的那个版本是什么？","candidates":[{"text":"关于接口版本号这件事，会上也提过一嘴，但那次的说法是64GB，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于接口版本号这件事，会上也提过一嘴，但那次的说法是夜间 02:00，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于接口版本号这件事，会上也提过一嘴，但那次的说法是10.20.3.7:5432，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于接口版本号这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认v3.14.2，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3],"positive_index":3,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["v3.14.2"],"hop_count":1,"answer":"v3.14.2"}}
|
||||
{"id":"realistic-unknown_attribute-00023","group_id":"realistic-unknown_attribute-00023","source":"realistic_memory_eval","family":"unknown_attribute","query":"我写库连的是哪里？","candidates":[{"text":"我的应急联系人是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00024","group_id":"realistic-multi_entity-00024","source":"realistic_memory_eval","family":"multi_entity","query":"我们组那位张三出问题该喊谁？","candidates":[{"text":"项目 Alpha出事找 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我出事找 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"张三的应急联系人是 91.5%。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["91.5%"],"hop_count":1,"answer":"91.5%","attribute":"应急联系人"}}
|
||||
{"id":"realistic-alias_paraphrase-00025","group_id":"realistic-alias_paraphrase-00025","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我用的那台是什么款？","candidates":[{"text":"我常驻机房在 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我手上那台机器是 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 v3.14.2。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["B3-204"],"hop_count":1,"answer":"B3-204"}}
|
||||
{"id":"realistic-update_conflict-00026","group_id":"realistic-update_conflict-00026","source":"realistic_memory_eval","family":"update_conflict","query":"我出问题该喊谁？","candidates":[{"text":"我的应急联系人是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我出事找 78%。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[4],"positive_index":4,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["78%"],"hop_count":1,"answer":"78%","attribute":"应急联系人","superseded_value":"每周三 22:00"}}
|
||||
{"id":"realistic-multi_hop-00027","group_id":"realistic-multi_hop-00027","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-4KN3KZ 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-4KN3KZ。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-HFV7EH 对应的值班人是 值班人-4124。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-24CF7B 对应的值班人是 值班人-5672。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-XWACAL 对应的值班人是 值班人-6064。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0,1],"positive_index":0,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00028","group_id":"realistic-near_miss-00028","source":"realistic_memory_eval","family":"near_miss","query":"我随身那台设备的末几位数字是啥？","candidates":[{"text":"我的手机尾号是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我的座机尾号是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的接口版本号是 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 李工 139****3344。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["v3.14.2"],"hop_count":1,"answer":"v3.14.2","attribute":"手机尾号"}}
|
||||
{"id":"realistic-noise_context-00029","group_id":"realistic-noise_context-00029","source":"realistic_memory_eval","family":"noise_context","query":"我出问题该喊谁？","candidates":[{"text":"这是普通对话噪声：验证用户-324536 暂时提到一个无关编号 N-5BBKKXZ8，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-909673 暂时提到一个无关编号 N-5W23N5GF，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-672128 暂时提到一个无关编号 N-P63KW9MK，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-475734 暂时提到一个无关编号 N-VAWA7DFW，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-375109 暂时提到一个无关编号 N-EH2N94PQ，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-129515 暂时提到一个无关编号 N-J9H82CDY，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-961176 暂时提到一个无关编号 N-KAGWWDC6，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-496074 暂时提到一个无关编号 N-BDUDDAZX，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[7],"positive_index":7,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["分机 8821"],"hop_count":1,"answer":"分机 8821"}}
|
||||
{"id":"realistic-long_fact-00030","group_id":"realistic-long_fact-00030","source":"realistic_memory_eval","family":"long_fact","query":"我到多少会触发报警？","candidates":[{"text":"关于告警阈值这件事，会上也提过一嘴，但那次的说法是64GB，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于告警阈值这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认李工 139****3344，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""},{"text":"关于告警阈值这件事，会上也提过一嘴，但那次的说法是91.5%，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于告警阈值这件事，会上也提过一嘴，但那次的说法是91.5%，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["李工 139****3344"],"hop_count":1,"answer":"李工 139****3344"}}
|
||||
{"id":"realistic-unknown_attribute-00031","group_id":"realistic-unknown_attribute-00031","source":"realistic_memory_eval","family":"unknown_attribute","query":"我数据落在哪一个地址？","candidates":[{"text":"我的应急联系人是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 王工 138****6621。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 王工 138****6621。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00032","group_id":"realistic-multi_entity-00032","source":"realistic_memory_eval","family":"multi_entity","query":"我自己出问题该喊谁？","candidates":[{"text":"我的应急联系人是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"项目 Alpha出事找 王工 138****6621。","kind":"fact","entity":"user","attribute":""},{"text":"李工出事找 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["A-7719"],"hop_count":1,"answer":"A-7719","attribute":"应急联系人"}}
|
||||
{"id":"realistic-alias_paraphrase-00033","group_id":"realistic-alias_paraphrase-00033","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我常年落脚在哪儿？","candidates":[{"text":"我平时待得最久的那座城是 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 王工 138****6621。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 A-7719。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["李工 139****3344"],"hop_count":1,"answer":"李工 139****3344"}}
|
||||
{"id":"realistic-update_conflict-00034","group_id":"realistic-update_conflict-00034","source":"realistic_memory_eval","family":"update_conflict","query":"我缓冲配了多少？","candidates":[{"text":"我的缓存容量是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我缓存开到 v2.8.0。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 128GB。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["v2.8.0"],"hop_count":1,"answer":"v2.8.0","attribute":"缓存容量","superseded_value":"78%"}}
|
||||
{"id":"realistic-multi_hop-00035","group_id":"realistic-multi_hop-00035","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-B7DYFD 对应的值班人是 值班人-7494。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-N3RPAR 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-N3RPAR。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-NBEDPE 对应的值班人是 值班人-1363。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-ARZQED 对应的值班人是 值班人-2074。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1,2],"positive_index":1,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00036","group_id":"realistic-near_miss-00036","source":"realistic_memory_eval","family":"near_miss","query":"我家座机最后几位数字是啥？","candidates":[{"text":"我的办公城市是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我的接口版本号是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的常住城市是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 夜间 02:00。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["夜间 02:00"],"hop_count":1,"answer":"夜间 02:00","attribute":"常住城市"}}
|
||||
{"id":"realistic-noise_context-00037","group_id":"realistic-noise_context-00037","source":"realistic_memory_eval","family":"noise_context","query":"我出问题该喊谁？","candidates":[{"text":"这是普通对话噪声：验证用户-033692 暂时提到一个无关编号 N-F82JLHQG，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-095735 暂时提到一个无关编号 N-ZK8Y6KBE，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-806056 暂时提到一个无关编号 N-DRFBAV2Q，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-054521 暂时提到一个无关编号 N-5B62H7W6，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-512728 暂时提到一个无关编号 N-XS252B9W，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-983485 暂时提到一个无关编号 N-D5YGVQNG，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-286058 暂时提到一个无关编号 N-PU3CQMGD，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-858456 暂时提到一个无关编号 N-QW3BANR7，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 v2.8.0。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[8],"positive_index":8,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["v2.8.0"],"hop_count":1,"answer":"v2.8.0"}}
|
||||
{"id":"realistic-long_fact-00038","group_id":"realistic-long_fact-00038","source":"realistic_memory_eval","family":"long_fact","query":"我哪条线一破就叫？","candidates":[{"text":"关于告警阈值这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认B3-204，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""},{"text":"关于告警阈值这件事，会上也提过一嘴，但那次的说法是v2.8.0，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于告警阈值这件事，会上也提过一嘴，但那次的说法是B3-204，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于告警阈值这件事，会上也提过一嘴，但那次的说法是128GB，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["B3-204"],"hop_count":1,"answer":"B3-204"}}
|
||||
{"id":"realistic-unknown_attribute-00039","group_id":"realistic-unknown_attribute-00039","source":"realistic_memory_eval","family":"unknown_attribute","query":"我一般什么时候做上线？","candidates":[{"text":"我的缓存容量是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 王工 138****6621。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00040","group_id":"realistic-multi_entity-00040","source":"realistic_memory_eval","family":"multi_entity","query":"Alpha出问题该喊谁？","candidates":[{"text":"我出事找 v2.8.0。","kind":"fact","entity":"user","attribute":""},{"text":"客户 Beta出事找 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"项目 Alpha的应急联系人是 v2.8.0。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["v2.8.0"],"hop_count":1,"answer":"v2.8.0","attribute":"应急联系人"}}
|
||||
{"id":"realistic-alias_paraphrase-00041","group_id":"realistic-alias_paraphrase-00041","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我那个数字是多少？","candidates":[{"text":"我的值班电话是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我每天睡醒最早看的那个数字是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 v2.8.0。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3],"positive_index":3,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["夜间 02:00"],"hop_count":1,"answer":"夜间 02:00"}}
|
||||
{"id":"realistic-update_conflict-00042","group_id":"realistic-update_conflict-00042","source":"realistic_memory_eval","family":"update_conflict","query":"我到多少会触发报警？","candidates":[{"text":"我的告警阈值是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我超过 v3.14.2 就报警。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 B-3310。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["v3.14.2"],"hop_count":1,"answer":"v3.14.2","attribute":"告警阈值","superseded_value":"10.20.9.1:6379"}}
|
||||
{"id":"realistic-multi_hop-00043","group_id":"realistic-multi_hop-00043","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-TRZ9QY 对应的值班人是 值班人-4384。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-H54FMF 对应的值班人是 值班人-4858。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-3A962Z 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-9JRMG3 对应的值班人是 值班人-7528。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-3A962Z。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2,4],"positive_index":2,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00044","group_id":"realistic-near_miss-00044","source":"realistic_memory_eval","family":"near_miss","query":"我家座机最后几位数字是啥？","candidates":[{"text":"我的应急联系人是 64GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的项目代号是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我的客户代号是 分机 8821。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3],"positive_index":3,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["分机 8821"],"hop_count":1,"answer":"分机 8821","attribute":"客户代号"}}
|
||||
{"id":"realistic-noise_context-00045","group_id":"realistic-noise_context-00045","source":"realistic_memory_eval","family":"noise_context","query":"我到多少会触发报警？","candidates":[{"text":"这是普通对话噪声：验证用户-483522 暂时提到一个无关编号 N-VM9XDGHQ，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-170445 暂时提到一个无关编号 N-6TKPZ7ZK，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-901205 暂时提到一个无关编号 N-PZKVUT72，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-498813 暂时提到一个无关编号 N-3V2FQ566，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-359672 暂时提到一个无关编号 N-S82NF75D，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-941011 暂时提到一个无关编号 N-7KF3P8EG，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-880148 暂时提到一个无关编号 N-U2XQ57WL，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-679421 暂时提到一个无关编号 N-NV53GTEA，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[5],"positive_index":5,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["A-7719"],"hop_count":1,"answer":"A-7719"}}
|
||||
{"id":"realistic-long_fact-00046","group_id":"realistic-long_fact-00046","source":"realistic_memory_eval","family":"long_fact","query":"我数据落在哪一个地址？","candidates":[{"text":"关于主库地址这件事，会上也提过一嘴，但那次的说法是v3.14.2，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于主库地址这件事，会上也提过一嘴，但那次的说法是A-7719，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于主库地址这件事，会上也提过一嘴，但那次的说法是B3-204，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于主库地址这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认10.20.9.1:6379，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3],"positive_index":3,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["10.20.9.1:6379"],"hop_count":1,"answer":"10.20.9.1:6379"}}
|
||||
{"id":"realistic-unknown_attribute-00047","group_id":"realistic-unknown_attribute-00047","source":"realistic_memory_eval","family":"unknown_attribute","query":"我有急事按哪个号找人？","candidates":[{"text":"我常驻机房在 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 A-7719。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00048","group_id":"realistic-multi_entity-00048","source":"realistic_memory_eval","family":"multi_entity","query":"Alpha缓冲配了多少？","candidates":[{"text":"李工缓存开到 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"项目 Alpha的缓存容量是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"张三缓存开到 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["B-3310"],"hop_count":1,"answer":"B-3310","attribute":"缓存容量"}}
|
||||
{"id":"realistic-alias_paraphrase-00049","group_id":"realistic-alias_paraphrase-00049","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我常年落脚在哪儿？","candidates":[{"text":"我常驻机房在 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我平时待得最久的那座城是 128GB。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[4],"positive_index":4,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["128GB"],"hop_count":1,"answer":"128GB"}}
|
||||
{"id":"realistic-update_conflict-00050","group_id":"realistic-update_conflict-00050","source":"realistic_memory_eval","family":"update_conflict","query":"我一般什么时候做上线？","candidates":[{"text":"我的发布窗口是 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 64GB。","kind":"fact","entity":"user","attribute":""},{"text":"我固定 B3-204 做上线。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 B3-204。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3],"positive_index":3,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["B3-204"],"hop_count":1,"answer":"B3-204","attribute":"发布窗口","superseded_value":"A-7719"}}
|
||||
{"id":"realistic-multi_hop-00051","group_id":"realistic-multi_hop-00051","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-VXCUTN 对应的值班人是 值班人-6159。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-TCDQ7V 对应的值班人是 值班人-1723。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-FLZEGX 对应的值班人是 值班人-5111。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-4Z8UBN 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-4Z8UBN。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3,4],"positive_index":3,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00052","group_id":"realistic-near_miss-00052","source":"realistic_memory_eval","family":"near_miss","query":"我随身那台设备的末几位数字是啥？","candidates":[{"text":"我的常住城市是 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的办公城市是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 78%。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["B-3310"],"hop_count":1,"answer":"B-3310","attribute":"办公城市"}}
|
||||
{"id":"realistic-noise_context-00053","group_id":"realistic-noise_context-00053","source":"realistic_memory_eval","family":"noise_context","query":"我平时在哪间屋子干活？","candidates":[{"text":"这是普通对话噪声：验证用户-282856 暂时提到一个无关编号 N-46XLUAEJ，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-029255 暂时提到一个无关编号 N-8TQUFEMZ，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-321060 暂时提到一个无关编号 N-AMPJX775，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-281765 暂时提到一个无关编号 N-LN8LM3EF，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-427871 暂时提到一个无关编号 N-6SNTL27G，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-124757 暂时提到一个无关编号 N-UTGA632L，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-569690 暂时提到一个无关编号 N-XSGXUGDZ，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-248927 暂时提到一个无关编号 N-KL95UYUA，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[5],"positive_index":5,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["10.20.3.7:5432"],"hop_count":1,"answer":"10.20.3.7:5432"}}
|
||||
{"id":"realistic-long_fact-00054","group_id":"realistic-long_fact-00054","source":"realistic_memory_eval","family":"long_fact","query":"我一般什么时候做上线？","candidates":[{"text":"关于发布窗口这件事，会上也提过一嘴，但那次的说法是78%，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于发布窗口这件事，会上也提过一嘴，但那次的说法是v2.8.0，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于发布窗口这件事，会上也提过一嘴，但那次的说法是王工 138****6621，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于发布窗口这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认v3.14.2，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3],"positive_index":3,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["v3.14.2"],"hop_count":1,"answer":"v3.14.2"}}
|
||||
{"id":"realistic-unknown_attribute-00055","group_id":"realistic-unknown_attribute-00055","source":"realistic_memory_eval","family":"unknown_attribute","query":"我现在对接的是哪一版？","candidates":[{"text":"我的告警阈值是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 王工 138****6621。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 v2.8.0。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00056","group_id":"realistic-multi_entity-00056","source":"realistic_memory_eval","family":"multi_entity","query":"我们组那位张三现在对接的是哪一版？","candidates":[{"text":"张三对接的接口版本号是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"李工目前跑的是 10.20.3.7:5432 版本接口。","kind":"fact","entity":"user","attribute":""},{"text":"我目前跑的是 v3.14.2 版本接口。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["78%"],"hop_count":1,"answer":"78%","attribute":"接口版本号"}}
|
||||
{"id":"realistic-alias_paraphrase-00057","group_id":"realistic-alias_paraphrase-00057","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我用的那台是什么款？","candidates":[{"text":"我手上那台机器是 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 78%。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["B3-204"],"hop_count":1,"answer":"B3-204"}}
|
||||
{"id":"realistic-update_conflict-00058","group_id":"realistic-update_conflict-00058","source":"realistic_memory_eval","family":"update_conflict","query":"我挑哪个时段变更？","candidates":[{"text":"我的发布窗口是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"我固定 夜间 02:00 做上线。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 v2.8.0。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 李工 139****3344。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["夜间 02:00"],"hop_count":1,"answer":"夜间 02:00","attribute":"发布窗口","superseded_value":"李工 139****3344"}}
|
||||
{"id":"realistic-multi_hop-00059","group_id":"realistic-multi_hop-00059","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-ASMHJ7 对应的值班人是 值班人-4620。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-WUN8SL 对应的值班人是 值班人-7348。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-7RE8RP 对应的值班人是 值班人-9179。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-H2DKPX 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-H2DKPX。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3,4],"positive_index":3,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00060","group_id":"realistic-near_miss-00060","source":"realistic_memory_eval","family":"near_miss","query":"我家座机最后几位数字是啥？","candidates":[{"text":"我的座机尾号是 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"我的常驻机房是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我的手机尾号是 王工 138****6621。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 78%。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["李工 139****3344"],"hop_count":1,"answer":"李工 139****3344","attribute":"座机尾号"}}
|
||||
{"id":"realistic-noise_context-00061","group_id":"realistic-noise_context-00061","source":"realistic_memory_eval","family":"noise_context","query":"我一般什么时候做上线？","candidates":[{"text":"这是普通对话噪声：验证用户-718842 暂时提到一个无关编号 N-HDG86AHG，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-141787 暂时提到一个无关编号 N-D9X34AVC，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-902078 暂时提到一个无关编号 N-PCTFECWF，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-973548 暂时提到一个无关编号 N-6J8QAGZV，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-337657 暂时提到一个无关编号 N-694NKRQH，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-466038 暂时提到一个无关编号 N-Z4YE82QN，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-052668 暂时提到一个无关编号 N-KUNMRYV5，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-833362 暂时提到一个无关编号 N-7END4NUG，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[5],"positive_index":5,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["91.5%"],"hop_count":1,"answer":"91.5%"}}
|
||||
{"id":"realistic-long_fact-00062","group_id":"realistic-long_fact-00062","source":"realistic_memory_eval","family":"long_fact","query":"我写库连的是哪里？","candidates":[{"text":"关于主库地址这件事，会上也提过一嘴，但那次的说法是128GB，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于主库地址这件事，会上也提过一嘴，但那次的说法是A-7719，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于主库地址这件事，会上也提过一嘴，但那次的说法是10.20.9.1:6379，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于主库地址这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认78%，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3],"positive_index":3,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["78%"],"hop_count":1,"answer":"78%"}}
|
||||
{"id":"realistic-unknown_attribute-00063","group_id":"realistic-unknown_attribute-00063","source":"realistic_memory_eval","family":"unknown_attribute","query":"我挑哪个时段变更？","candidates":[{"text":"我的告警阈值是 v2.8.0。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 B3-204。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00064","group_id":"realistic-multi_entity-00064","source":"realistic_memory_eval","family":"multi_entity","query":"我自己一般什么时候做上线？","candidates":[{"text":"我的发布窗口是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"客户 Beta固定 128GB 做上线。","kind":"fact","entity":"user","attribute":""},{"text":"项目 Alpha固定 夜间 02:00 做上线。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["v3.14.2"],"hop_count":1,"answer":"v3.14.2","attribute":"发布窗口"}}
|
||||
{"id":"realistic-alias_paraphrase-00065","group_id":"realistic-alias_paraphrase-00065","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我那个数字是多少？","candidates":[{"text":"我的发布窗口是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我每天睡醒最早看的那个数字是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 v2.8.0。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["B-3310"],"hop_count":1,"answer":"B-3310"}}
|
||||
{"id":"realistic-update_conflict-00066","group_id":"realistic-update_conflict-00066","source":"realistic_memory_eval","family":"update_conflict","query":"我轮班时该拨什么号码？","candidates":[{"text":"我的值班电话是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我留的值班号码为 每周三 22:00。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[4],"positive_index":4,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["每周三 22:00"],"hop_count":1,"answer":"每周三 22:00","attribute":"值班电话","superseded_value":"分机 8821"}}
|
||||
{"id":"realistic-multi_hop-00067","group_id":"realistic-multi_hop-00067","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-2RGQNH 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-2RGQNH。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-9FYNEZ 对应的值班人是 值班人-1996。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-58DPTU 对应的值班人是 值班人-2211。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-3G2HKH 对应的值班人是 值班人-8592。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0,1],"positive_index":0,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00068","group_id":"realistic-near_miss-00068","source":"realistic_memory_eval","family":"near_miss","query":"我家座机最后几位数字是啥？","candidates":[{"text":"我的客户代号是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"我的接口版本号是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我的项目代号是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 李工 139****3344。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["分机 8821"],"hop_count":1,"answer":"分机 8821","attribute":"客户代号"}}
|
||||
{"id":"realistic-noise_context-00069","group_id":"realistic-noise_context-00069","source":"realistic_memory_eval","family":"noise_context","query":"我一般什么时候做上线？","candidates":[{"text":"这是普通对话噪声：验证用户-532970 暂时提到一个无关编号 N-XYVL3XH9，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-091781 暂时提到一个无关编号 N-9HJ6VRZR，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-486342 暂时提到一个无关编号 N-4NJW2H9N，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-865272 暂时提到一个无关编号 N-V7DA2VUJ，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-235383 暂时提到一个无关编号 N-XYMG55ZW，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-477024 暂时提到一个无关编号 N-MA3GDW7N，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-080799 暂时提到一个无关编号 N-N3T5JPK8，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-933426 暂时提到一个无关编号 N-P29VT469，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[7],"positive_index":7,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["v3.14.2"],"hop_count":1,"answer":"v3.14.2"}}
|
||||
{"id":"realistic-long_fact-00070","group_id":"realistic-long_fact-00070","source":"realistic_memory_eval","family":"long_fact","query":"我轮班时该拨什么号码？","candidates":[{"text":"关于值班电话这件事，会上也提过一嘴，但那次的说法是每周三 22:00，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于值班电话这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认分机 8821，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""},{"text":"关于值班电话这件事，会上也提过一嘴，但那次的说法是每周三 22:00，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于值班电话这件事，会上也提过一嘴，但那次的说法是A-7719，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["分机 8821"],"hop_count":1,"answer":"分机 8821"}}
|
||||
{"id":"realistic-unknown_attribute-00071","group_id":"realistic-unknown_attribute-00071","source":"realistic_memory_eval","family":"unknown_attribute","query":"我缓冲配了多少？","candidates":[{"text":"我的主库地址是 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 分机 8821。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00072","group_id":"realistic-multi_entity-00072","source":"realistic_memory_eval","family":"multi_entity","query":"Beta有急事按哪个号找人？","candidates":[{"text":"我留的值班号码为 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"李工留的值班号码为 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"客户 Beta的值班电话是 分机 8821。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["分机 8821"],"hop_count":1,"answer":"分机 8821","attribute":"值班电话"}}
|
||||
{"id":"realistic-alias_paraphrase-00073","group_id":"realistic-alias_paraphrase-00073","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我早上那杯是什么口味？","candidates":[{"text":"我的告警阈值是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 64GB。","kind":"fact","entity":"user","attribute":""},{"text":"我每天早上灌下去的那杯偏爱 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 B-3310。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["91.5%"],"hop_count":1,"answer":"91.5%"}}
|
||||
{"id":"realistic-update_conflict-00074","group_id":"realistic-update_conflict-00074","source":"realistic_memory_eval","family":"update_conflict","query":"我有急事按哪个号找人？","candidates":[{"text":"我的值班电话是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我留的值班号码为 分机 8821。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[4],"positive_index":4,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["分机 8821"],"hop_count":1,"answer":"分机 8821","attribute":"值班电话","superseded_value":"A-7719"}}
|
||||
{"id":"realistic-multi_hop-00075","group_id":"realistic-multi_hop-00075","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"张三负责的那个项目的内部编号是 REF-WLFBUN。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-DK56DD 对应的值班人是 值班人-6619。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-6K39V9 对应的值班人是 值班人-4912。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-62PNDR 对应的值班人是 值班人-9778。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-WLFBUN 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0,4],"positive_index":0,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00076","group_id":"realistic-near_miss-00076","source":"realistic_memory_eval","family":"near_miss","query":"我随身那台设备的末几位数字是啥？","candidates":[{"text":"我的发布窗口是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我的宿舍楼层是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我的工位楼层是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 B-3310。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["128GB"],"hop_count":1,"answer":"128GB","attribute":"工位楼层"}}
|
||||
{"id":"realistic-noise_context-00077","group_id":"realistic-noise_context-00077","source":"realistic_memory_eval","family":"noise_context","query":"我固定在哪个位置办公？","candidates":[{"text":"我常驻机房在 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-069004 暂时提到一个无关编号 N-4QGAC2N4，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-715686 暂时提到一个无关编号 N-76BE6JPF，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-639378 暂时提到一个无关编号 N-MQKDSH4K，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-070195 暂时提到一个无关编号 N-8W5U6T2H，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-584184 暂时提到一个无关编号 N-JDX6DT8J，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-100606 暂时提到一个无关编号 N-6QY38SXS，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-113325 暂时提到一个无关编号 N-MEEWAFUS，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-481246 暂时提到一个无关编号 N-W4B92AT4，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["分机 8821"],"hop_count":1,"answer":"分机 8821"}}
|
||||
{"id":"realistic-long_fact-00078","group_id":"realistic-long_fact-00078","source":"realistic_memory_eval","family":"long_fact","query":"我缓冲配了多少？","candidates":[{"text":"关于缓存容量这件事，会上也提过一嘴，但那次的说法是夜间 02:00，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于缓存容量这件事，会上也提过一嘴，但那次的说法是10.20.3.7:5432，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于缓存容量这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认v2.8.0，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""},{"text":"关于缓存容量这件事，会上也提过一嘴，但那次的说法是分机 8821，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["v2.8.0"],"hop_count":1,"answer":"v2.8.0"}}
|
||||
{"id":"realistic-unknown_attribute-00079","group_id":"realistic-unknown_attribute-00079","source":"realistic_memory_eval","family":"unknown_attribute","query":"我平时在哪间屋子干活？","candidates":[{"text":"我的应急联系人是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 v2.8.0。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00080","group_id":"realistic-multi_entity-00080","source":"realistic_memory_eval","family":"multi_entity","query":"Beta缓冲配了多少？","candidates":[{"text":"项目 Alpha缓存开到 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我缓存开到 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"客户 Beta的缓存容量是 128GB。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["128GB"],"hop_count":1,"answer":"128GB","attribute":"缓存容量"}}
|
||||
{"id":"realistic-alias_paraphrase-00081","group_id":"realistic-alias_paraphrase-00081","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我早上那杯是什么口味？","candidates":[{"text":"我的值班电话是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 64GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我每天早上灌下去的那杯偏爱 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 每周三 22:00。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3],"positive_index":3,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["B-3310"],"hop_count":1,"answer":"B-3310"}}
|
||||
{"id":"realistic-update_conflict-00082","group_id":"realistic-update_conflict-00082","source":"realistic_memory_eval","family":"update_conflict","query":"我平时在哪间屋子干活？","candidates":[{"text":"我平时待的机房是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 王工 138****6621。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 v3.14.2。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["A-7719"],"hop_count":1,"answer":"A-7719","attribute":"常驻机房","superseded_value":"王工 138****6621"}}
|
||||
{"id":"realistic-multi_hop-00083","group_id":"realistic-multi_hop-00083","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"张三负责的那个项目的内部编号是 REF-8JYSKT。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-C8P8JT 对应的值班人是 值班人-1586。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-VX3KXH 对应的值班人是 值班人-7768。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-N46TFA 对应的值班人是 值班人-3913。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-8JYSKT 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0,4],"positive_index":0,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00084","group_id":"realistic-near_miss-00084","source":"realistic_memory_eval","family":"near_miss","query":"我随身那台设备的末几位数字是啥？","candidates":[{"text":"我的告警阈值是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的座机尾号是 王工 138****6621。","kind":"fact","entity":"user","attribute":""},{"text":"我的手机尾号是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3],"positive_index":3,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["10.20.3.7:5432"],"hop_count":1,"answer":"10.20.3.7:5432","attribute":"手机尾号"}}
|
||||
{"id":"realistic-noise_context-00085","group_id":"realistic-noise_context-00085","source":"realistic_memory_eval","family":"noise_context","query":"我出问题该喊谁？","candidates":[{"text":"这是普通对话噪声：验证用户-157710 暂时提到一个无关编号 N-D28BW56M，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-181438 暂时提到一个无关编号 N-HSUMDGQ8，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-119197 暂时提到一个无关编号 N-3Z2FTKUZ，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-210199 暂时提到一个无关编号 N-EQV4GGZJ，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-876469 暂时提到一个无关编号 N-8E3DDQAN，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-033668 暂时提到一个无关编号 N-EVQHD45X，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-866380 暂时提到一个无关编号 N-FNRNAPSM，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-405350 暂时提到一个无关编号 N-BMSRG6VG，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[5],"positive_index":5,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["分机 8821"],"hop_count":1,"answer":"分机 8821"}}
|
||||
{"id":"realistic-long_fact-00086","group_id":"realistic-long_fact-00086","source":"realistic_memory_eval","family":"long_fact","query":"我哪条线一破就叫？","candidates":[{"text":"关于告警阈值这件事，会上也提过一嘴，但那次的说法是分机 8821，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于告警阈值这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认每周三 22:00，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""},{"text":"关于告警阈值这件事，会上也提过一嘴，但那次的说法是每周三 22:00，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于告警阈值这件事，会上也提过一嘴，但那次的说法是夜间 02:00，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["每周三 22:00"],"hop_count":1,"answer":"每周三 22:00"}}
|
||||
{"id":"realistic-unknown_attribute-00087","group_id":"realistic-unknown_attribute-00087","source":"realistic_memory_eval","family":"unknown_attribute","query":"我平时在哪间屋子干活？","candidates":[{"text":"我的告警阈值是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 B-3310。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00088","group_id":"realistic-multi_entity-00088","source":"realistic_memory_eval","family":"multi_entity","query":"运维那位李工平时在哪间屋子干活？","candidates":[{"text":"张三平时待的机房是 王工 138****6621。","kind":"fact","entity":"user","attribute":""},{"text":"项目 Alpha平时待的机房是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"李工常驻机房在 分机 8821。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["分机 8821"],"hop_count":1,"answer":"分机 8821","attribute":"常驻机房"}}
|
||||
{"id":"realistic-alias_paraphrase-00089","group_id":"realistic-alias_paraphrase-00089","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我常年落脚在哪儿？","candidates":[{"text":"我平时待得最久的那座城是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 王工 138****6621。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["分机 8821"],"hop_count":1,"answer":"分机 8821"}}
|
||||
{"id":"realistic-update_conflict-00090","group_id":"realistic-update_conflict-00090","source":"realistic_memory_eval","family":"update_conflict","query":"我平时在哪间屋子干活？","candidates":[{"text":"我平时待的机房是 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 v2.8.0。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 64GB。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 64GB。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["B3-204"],"hop_count":1,"answer":"B3-204","attribute":"常驻机房","superseded_value":"v2.8.0"}}
|
||||
{"id":"realistic-multi_hop-00091","group_id":"realistic-multi_hop-00091","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-VDUPJA 对应的值班人是 值班人-4539。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-2XL68B。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-2XL68B 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-FLEQLG 对应的值班人是 值班人-4366。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-UVZ2WM 对应的值班人是 值班人-4145。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1,2],"positive_index":1,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00092","group_id":"realistic-near_miss-00092","source":"realistic_memory_eval","family":"near_miss","query":"我家座机最后几位数字是啥？","candidates":[{"text":"我的缓存容量是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的手机尾号是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我的座机尾号是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 128GB。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["A-7719"],"hop_count":1,"answer":"A-7719","attribute":"座机尾号"}}
|
||||
{"id":"realistic-noise_context-00093","group_id":"realistic-noise_context-00093","source":"realistic_memory_eval","family":"noise_context","query":"我轮班时该拨什么号码？","candidates":[{"text":"这是普通对话噪声：验证用户-223399 暂时提到一个无关编号 N-MRWFVSCN，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-585544 暂时提到一个无关编号 N-NXDWKUSV，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-871137 暂时提到一个无关编号 N-U39K4J69，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-482041 暂时提到一个无关编号 N-NTBPJVCY，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-293078 暂时提到一个无关编号 N-EAY3QE5E，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-640476 暂时提到一个无关编号 N-5KWNWBYC，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-268520 暂时提到一个无关编号 N-VZ4BH9MK，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-581858 暂时提到一个无关编号 N-NYV84UD4，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[6],"positive_index":6,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["128GB"],"hop_count":1,"answer":"128GB"}}
|
||||
{"id":"realistic-long_fact-00094","group_id":"realistic-long_fact-00094","source":"realistic_memory_eval","family":"long_fact","query":"我出问题该喊谁？","candidates":[{"text":"关于应急联系人这件事，会上也提过一嘴，但那次的说法是10.20.9.1:6379，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于应急联系人这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认v2.8.0，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""},{"text":"关于应急联系人这件事，会上也提过一嘴，但那次的说法是分机 8821，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于应急联系人这件事，会上也提过一嘴，但那次的说法是10.20.3.7:5432，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["v2.8.0"],"hop_count":1,"answer":"v2.8.0"}}
|
||||
{"id":"realistic-unknown_attribute-00095","group_id":"realistic-unknown_attribute-00095","source":"realistic_memory_eval","family":"unknown_attribute","query":"我缓冲配了多少？","candidates":[{"text":"我的发布窗口是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 64GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00096","group_id":"realistic-multi_entity-00096","source":"realistic_memory_eval","family":"multi_entity","query":"Alpha哪条线一破就叫？","candidates":[{"text":"我超过 10.20.9.1:6379 就报警。","kind":"fact","entity":"user","attribute":""},{"text":"张三超过 10.20.9.1:6379 就报警。","kind":"fact","entity":"user","attribute":""},{"text":"项目 Alpha的告警阈值是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["10.20.9.1:6379"],"hop_count":1,"answer":"10.20.9.1:6379","attribute":"告警阈值"}}
|
||||
{"id":"realistic-alias_paraphrase-00097","group_id":"realistic-alias_paraphrase-00097","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我那个数字是多少？","candidates":[{"text":"我常驻机房在 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 v2.8.0。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我每天睡醒最早看的那个数字是 91.5%。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[4],"positive_index":4,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["91.5%"],"hop_count":1,"answer":"91.5%"}}
|
||||
{"id":"realistic-update_conflict-00098","group_id":"realistic-update_conflict-00098","source":"realistic_memory_eval","family":"update_conflict","query":"我一般什么时候做上线？","candidates":[{"text":"我的发布窗口是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"我固定 10.20.3.7:5432 做上线。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[4],"positive_index":4,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["10.20.3.7:5432"],"hop_count":1,"answer":"10.20.3.7:5432","attribute":"发布窗口","superseded_value":"A-7719"}}
|
||||
{"id":"realistic-multi_hop-00099","group_id":"realistic-multi_hop-00099","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-RHUMBL 对应的值班人是 值班人-3926。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-QP5XFC 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-QP5XFC。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-HWDDAS 对应的值班人是 值班人-4075。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-G676CS 对应的值班人是 值班人-8553。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1,2],"positive_index":1,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00100","group_id":"realistic-near_miss-00100","source":"realistic_memory_eval","family":"near_miss","query":"我家座机最后几位数字是啥？","candidates":[{"text":"我的缓存容量是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我的办公城市是 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我的常住城市是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我的接口版本号是 91.5%。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["78%"],"hop_count":1,"answer":"78%","attribute":"常住城市"}}
|
||||
{"id":"realistic-noise_context-00101","group_id":"realistic-noise_context-00101","source":"realistic_memory_eval","family":"noise_context","query":"我联调用的那个版本是什么？","candidates":[{"text":"这是普通对话噪声：验证用户-152257 暂时提到一个无关编号 N-C8JEP7KJ，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-338407 暂时提到一个无关编号 N-BCMYSY27，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-617608 暂时提到一个无关编号 N-FW53QV5A，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-229720 暂时提到一个无关编号 N-C8NCSXQM，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-904020 暂时提到一个无关编号 N-AMZJCQZF，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-352729 暂时提到一个无关编号 N-8PFTN2L3，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-609377 暂时提到一个无关编号 N-3Q6TYQ23，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-079202 暂时提到一个无关编号 N-H33DYYEK，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["10.20.3.7:5432"],"hop_count":1,"answer":"10.20.3.7:5432"}}
|
||||
{"id":"realistic-long_fact-00102","group_id":"realistic-long_fact-00102","source":"realistic_memory_eval","family":"long_fact","query":"我写库连的是哪里？","candidates":[{"text":"关于主库地址这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认91.5%，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""},{"text":"关于主库地址这件事，会上也提过一嘴，但那次的说法是B-3310，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于主库地址这件事，会上也提过一嘴，但那次的说法是78%，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于主库地址这件事，会上也提过一嘴，但那次的说法是每周三 22:00，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["91.5%"],"hop_count":1,"answer":"91.5%"}}
|
||||
{"id":"realistic-unknown_attribute-00103","group_id":"realistic-unknown_attribute-00103","source":"realistic_memory_eval","family":"unknown_attribute","query":"我紧急情况下找谁接手？","candidates":[{"text":"我对接的接口版本号是 王工 138****6621。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 v3.14.2。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00104","group_id":"realistic-multi_entity-00104","source":"realistic_memory_eval","family":"multi_entity","query":"Beta 客户内存开到多大？","candidates":[{"text":"我缓存开到 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"客户 Beta的缓存容量是 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"项目 Alpha缓存开到 李工 139****3344。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["B3-204"],"hop_count":1,"answer":"B3-204","attribute":"缓存容量"}}
|
||||
{"id":"realistic-alias_paraphrase-00105","group_id":"realistic-alias_paraphrase-00105","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我用的那台是什么款？","candidates":[{"text":"我对接的接口版本号是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"我手上那台机器是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 王工 138****6621。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["每周三 22:00"],"hop_count":1,"answer":"每周三 22:00"}}
|
||||
{"id":"realistic-update_conflict-00106","group_id":"realistic-update_conflict-00106","source":"realistic_memory_eval","family":"update_conflict","query":"我联调用的那个版本是什么？","candidates":[{"text":"我目前跑的是 A-7719 版本接口。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 64GB。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 78%。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["A-7719"],"hop_count":1,"answer":"A-7719","attribute":"接口版本号","superseded_value":"10.20.9.1:6379"}}
|
||||
{"id":"realistic-multi_hop-00107","group_id":"realistic-multi_hop-00107","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-2B6LK8 对应的值班人是 值班人-9934。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-M4XWLY。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-M4XWLY 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-8UHLJE 对应的值班人是 值班人-7190。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-E5FJQ7 对应的值班人是 值班人-1296。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1,2],"positive_index":1,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00108","group_id":"realistic-near_miss-00108","source":"realistic_memory_eval","family":"near_miss","query":"我家座机最后几位数字是啥？","candidates":[{"text":"我的告警阈值是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的办公城市是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的常住城市是 夜间 02:00。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["夜间 02:00"],"hop_count":1,"answer":"夜间 02:00","attribute":"常住城市"}}
|
||||
{"id":"realistic-noise_context-00109","group_id":"realistic-noise_context-00109","source":"realistic_memory_eval","family":"noise_context","query":"我一般什么时候做上线？","candidates":[{"text":"这是普通对话噪声：验证用户-202308 暂时提到一个无关编号 N-EGW48KVQ，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-976098 暂时提到一个无关编号 N-7TJAZBA9，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-607141 暂时提到一个无关编号 N-HZE9LY34，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-155674 暂时提到一个无关编号 N-9VPWYCGP，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-940168 暂时提到一个无关编号 N-GHF4S3VB，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-906143 暂时提到一个无关编号 N-YL723N74，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-917908 暂时提到一个无关编号 N-V97QEUSB，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-709070 暂时提到一个无关编号 N-WCN245UH，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 v2.8.0。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[8],"positive_index":8,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["v2.8.0"],"hop_count":1,"answer":"v2.8.0"}}
|
||||
{"id":"realistic-long_fact-00110","group_id":"realistic-long_fact-00110","source":"realistic_memory_eval","family":"long_fact","query":"我固定在哪个位置办公？","candidates":[{"text":"关于常驻机房这件事，会上也提过一嘴，但那次的说法是v2.8.0，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于常驻机房这件事，会上也提过一嘴，但那次的说法是78%，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于常驻机房这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认10.20.3.7:5432，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""},{"text":"关于常驻机房这件事，会上也提过一嘴，但那次的说法是91.5%，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["10.20.3.7:5432"],"hop_count":1,"answer":"10.20.3.7:5432"}}
|
||||
{"id":"realistic-unknown_attribute-00111","group_id":"realistic-unknown_attribute-00111","source":"realistic_memory_eval","family":"unknown_attribute","query":"我内存开到多大？","candidates":[{"text":"我的值班电话是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 王工 138****6621。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00112","group_id":"realistic-multi_entity-00112","source":"realistic_memory_eval","family":"multi_entity","query":"Beta 客户写库连的是哪里？","candidates":[{"text":"我主库落在 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"张三主库落在 v2.8.0。","kind":"fact","entity":"user","attribute":""},{"text":"客户 Beta的主库地址是 64GB。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["64GB"],"hop_count":1,"answer":"64GB","attribute":"主库地址"}}
|
||||
{"id":"realistic-alias_paraphrase-00113","group_id":"realistic-alias_paraphrase-00113","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我用的那台是什么款？","candidates":[{"text":"我的值班电话是 王工 138****6621。","kind":"fact","entity":"user","attribute":""},{"text":"我手上那台机器是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 64GB。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["B-3310"],"hop_count":1,"answer":"B-3310"}}
|
||||
{"id":"realistic-update_conflict-00114","group_id":"realistic-update_conflict-00114","source":"realistic_memory_eval","family":"update_conflict","query":"我一般什么时候做上线？","candidates":[{"text":"我的发布窗口是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我固定 A-7719 做上线。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 A-7719。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["A-7719"],"hop_count":1,"answer":"A-7719","attribute":"发布窗口","superseded_value":"10.20.9.1:6379"}}
|
||||
{"id":"realistic-multi_hop-00115","group_id":"realistic-multi_hop-00115","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-C29CD3 对应的值班人是 值班人-7381。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-599UFF 对应的值班人是 值班人-7261。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-USL5MH 对应的值班人是 值班人-3212。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-YCJAD4 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-YCJAD4。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3,4],"positive_index":3,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00116","group_id":"realistic-near_miss-00116","source":"realistic_memory_eval","family":"near_miss","query":"我家座机最后几位数字是啥？","candidates":[{"text":"我的座机尾号是 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的手机尾号是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 王工 138****6621。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["91.5%"],"hop_count":1,"answer":"91.5%","attribute":"座机尾号"}}
|
||||
{"id":"realistic-noise_context-00117","group_id":"realistic-noise_context-00117","source":"realistic_memory_eval","family":"noise_context","query":"我现在对接的是哪一版？","candidates":[{"text":"这是普通对话噪声：验证用户-659650 暂时提到一个无关编号 N-CRVPDN8Y，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-681190 暂时提到一个无关编号 N-L7LHBV22，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-718810 暂时提到一个无关编号 N-Y5ZTGD2S，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-900555 暂时提到一个无关编号 N-AKAVEVST，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-030594 暂时提到一个无关编号 N-UXGVCLN6，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-048337 暂时提到一个无关编号 N-ESH2QFLJ，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-878933 暂时提到一个无关编号 N-P4KXE48V，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-558557 暂时提到一个无关编号 N-TFHDL2YH，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[6],"positive_index":6,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["A-7719"],"hop_count":1,"answer":"A-7719"}}
|
||||
{"id":"realistic-long_fact-00118","group_id":"realistic-long_fact-00118","source":"realistic_memory_eval","family":"long_fact","query":"我写库连的是哪里？","candidates":[{"text":"关于主库地址这件事，会上也提过一嘴，但那次的说法是v3.14.2，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于主库地址这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认10.20.3.7:5432，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""},{"text":"关于主库地址这件事，会上也提过一嘴，但那次的说法是B3-204，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于主库地址这件事，会上也提过一嘴，但那次的说法是128GB，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["10.20.3.7:5432"],"hop_count":1,"answer":"10.20.3.7:5432"}}
|
||||
{"id":"realistic-unknown_attribute-00119","group_id":"realistic-unknown_attribute-00119","source":"realistic_memory_eval","family":"unknown_attribute","query":"我现在对接的是哪一版？","candidates":[{"text":"我的告警阈值是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 64GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 王工 138****6621。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 夜间 02:00。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00120","group_id":"realistic-multi_entity-00120","source":"realistic_memory_eval","family":"multi_entity","query":"Beta联调用的那个版本是什么？","candidates":[{"text":"客户 Beta对接的接口版本号是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我目前跑的是 91.5% 版本接口。","kind":"fact","entity":"user","attribute":""},{"text":"张三目前跑的是 分机 8821 版本接口。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["A-7719"],"hop_count":1,"answer":"A-7719","attribute":"接口版本号"}}
|
||||
{"id":"realistic-alias_paraphrase-00121","group_id":"realistic-alias_paraphrase-00121","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我常年落脚在哪儿？","candidates":[{"text":"我的发布窗口是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我平时待得最久的那座城是 B3-204。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[4],"positive_index":4,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["B3-204"],"hop_count":1,"answer":"B3-204"}}
|
||||
{"id":"realistic-update_conflict-00122","group_id":"realistic-update_conflict-00122","source":"realistic_memory_eval","family":"update_conflict","query":"我平时在哪间屋子干活？","candidates":[{"text":"我常驻机房在 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"我平时待的机房是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 91.5%。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["每周三 22:00"],"hop_count":1,"answer":"每周三 22:00","attribute":"常驻机房","superseded_value":"李工 139****3344"}}
|
||||
{"id":"realistic-multi_hop-00123","group_id":"realistic-multi_hop-00123","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-GWKJBN 对应的值班人是 值班人-6776。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-HQTF5Y 对应的值班人是 值班人-2241。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-QQYXK3。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-X6QVBE 对应的值班人是 值班人-8471。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-QQYXK3 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2,4],"positive_index":2,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00124","group_id":"realistic-near_miss-00124","source":"realistic_memory_eval","family":"near_miss","query":"我随身那台设备的末几位数字是啥？","candidates":[{"text":"我的常住城市是 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我的办公城市是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["10.20.9.1:6379"],"hop_count":1,"answer":"10.20.9.1:6379","attribute":"办公城市"}}
|
||||
{"id":"realistic-noise_context-00125","group_id":"realistic-noise_context-00125","source":"realistic_memory_eval","family":"noise_context","query":"我有急事按哪个号找人？","candidates":[{"text":"这是普通对话噪声：验证用户-128186 暂时提到一个无关编号 N-2WGXUK9T，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-315898 暂时提到一个无关编号 N-28VFKE32，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-902861 暂时提到一个无关编号 N-QWRPLSJB，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-112775 暂时提到一个无关编号 N-BC97Q7SK，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-847628 暂时提到一个无关编号 N-HQPK5JJ4，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-267701 暂时提到一个无关编号 N-6GEJD37T，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-690794 暂时提到一个无关编号 N-FUGZA5LT，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-999215 暂时提到一个无关编号 N-Z2959YN4，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[6],"positive_index":6,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["A-7719"],"hop_count":1,"answer":"A-7719"}}
|
||||
{"id":"realistic-long_fact-00126","group_id":"realistic-long_fact-00126","source":"realistic_memory_eval","family":"long_fact","query":"我挑哪个时段变更？","candidates":[{"text":"关于发布窗口这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认v2.8.0，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""},{"text":"关于发布窗口这件事，会上也提过一嘴，但那次的说法是10.20.9.1:6379，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于发布窗口这件事，会上也提过一嘴，但那次的说法是每周三 22:00，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于发布窗口这件事，会上也提过一嘴，但那次的说法是王工 138****6621，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["v2.8.0"],"hop_count":1,"answer":"v2.8.0"}}
|
||||
{"id":"realistic-unknown_attribute-00127","group_id":"realistic-unknown_attribute-00127","source":"realistic_memory_eval","family":"unknown_attribute","query":"我平时在哪间屋子干活？","candidates":[{"text":"我的缓存容量是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 B-3310。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00128","group_id":"realistic-multi_entity-00128","source":"realistic_memory_eval","family":"multi_entity","query":"我自己到多少会触发报警？","candidates":[{"text":"张三超过 128GB 就报警。","kind":"fact","entity":"user","attribute":""},{"text":"项目 Alpha超过 v3.14.2 就报警。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 每周三 22:00。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["每周三 22:00"],"hop_count":1,"answer":"每周三 22:00","attribute":"告警阈值"}}
|
||||
{"id":"realistic-alias_paraphrase-00129","group_id":"realistic-alias_paraphrase-00129","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我那个数字是多少？","candidates":[{"text":"我每天睡醒最早看的那个数字是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 每周三 22:00。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["v3.14.2"],"hop_count":1,"answer":"v3.14.2"}}
|
||||
{"id":"realistic-update_conflict-00130","group_id":"realistic-update_conflict-00130","source":"realistic_memory_eval","family":"update_conflict","query":"我联调用的那个版本是什么？","candidates":[{"text":"我对接的接口版本号是 64GB。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我目前跑的是 64GB 版本接口。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[4],"positive_index":4,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["64GB"],"hop_count":1,"answer":"64GB","attribute":"接口版本号","superseded_value":"10.20.3.7:5432"}}
|
||||
{"id":"realistic-multi_hop-00131","group_id":"realistic-multi_hop-00131","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-8SAPNH 对应的值班人是 值班人-5315。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-DWTZT6 对应的值班人是 值班人-3904。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-FK8H5Y 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-FK8H5Y。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-GEBEX2 对应的值班人是 值班人-4442。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2,3],"positive_index":2,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00132","group_id":"realistic-near_miss-00132","source":"realistic_memory_eval","family":"near_miss","query":"我家座机最后几位数字是啥？","candidates":[{"text":"我的发布窗口是 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我的座机尾号是 王工 138****6621。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的手机尾号是 91.5%。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["王工 138****6621"],"hop_count":1,"answer":"王工 138****6621","attribute":"座机尾号"}}
|
||||
{"id":"realistic-noise_context-00133","group_id":"realistic-noise_context-00133","source":"realistic_memory_eval","family":"noise_context","query":"我平时在哪间屋子干活？","candidates":[{"text":"这是普通对话噪声：验证用户-599981 暂时提到一个无关编号 N-MGH7TC69，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-698983 暂时提到一个无关编号 N-8AG4CMQG，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-791230 暂时提到一个无关编号 N-UQ4JJNMR，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-567791 暂时提到一个无关编号 N-GHET8G7V，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-730644 暂时提到一个无关编号 N-SNULKEMM，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-948673 暂时提到一个无关编号 N-KLXLTQL4，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-912909 暂时提到一个无关编号 N-35KPCU46，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-171723 暂时提到一个无关编号 N-7PHP9GYH，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[7],"positive_index":7,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["夜间 02:00"],"hop_count":1,"answer":"夜间 02:00"}}
|
||||
{"id":"realistic-long_fact-00134","group_id":"realistic-long_fact-00134","source":"realistic_memory_eval","family":"long_fact","query":"我缓冲配了多少？","candidates":[{"text":"关于缓存容量这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认v3.14.2，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""},{"text":"关于缓存容量这件事，会上也提过一嘴，但那次的说法是64GB，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于缓存容量这件事，会上也提过一嘴，但那次的说法是王工 138****6621，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于缓存容量这件事，会上也提过一嘴，但那次的说法是王工 138****6621，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["v3.14.2"],"hop_count":1,"answer":"v3.14.2"}}
|
||||
{"id":"realistic-unknown_attribute-00135","group_id":"realistic-unknown_attribute-00135","source":"realistic_memory_eval","family":"unknown_attribute","query":"我紧急情况下找谁接手？","candidates":[{"text":"我对接的接口版本号是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 64GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 128GB。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00136","group_id":"realistic-multi_entity-00136","source":"realistic_memory_eval","family":"multi_entity","query":"Alpha紧急情况下找谁接手？","candidates":[{"text":"李工出事找 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"项目 Alpha的应急联系人是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"张三出事找 每周三 22:00。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["128GB"],"hop_count":1,"answer":"128GB","attribute":"应急联系人"}}
|
||||
{"id":"realistic-alias_paraphrase-00137","group_id":"realistic-alias_paraphrase-00137","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我用的那台是什么款？","candidates":[{"text":"我的发布窗口是 64GB。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我手上那台机器是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 64GB。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3],"positive_index":3,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["分机 8821"],"hop_count":1,"answer":"分机 8821"}}
|
||||
{"id":"realistic-update_conflict-00138","group_id":"realistic-update_conflict-00138","source":"realistic_memory_eval","family":"update_conflict","query":"我现在对接的是哪一版？","candidates":[{"text":"我目前跑的是 B-3310 版本接口。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 李工 139****3344。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["B-3310"],"hop_count":1,"answer":"B-3310","attribute":"接口版本号","superseded_value":"夜间 02:00"}}
|
||||
{"id":"realistic-multi_hop-00139","group_id":"realistic-multi_hop-00139","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-BNWFND 对应的值班人是 值班人-2547。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-D6YASQ 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-D6YASQ。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-WJEE5U 对应的值班人是 值班人-1020。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-ZUQXNN 对应的值班人是 值班人-1752。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1,2],"positive_index":1,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00140","group_id":"realistic-near_miss-00140","source":"realistic_memory_eval","family":"near_miss","query":"我随身那台设备的末几位数字是啥？","candidates":[{"text":"我的办公城市是 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 64GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的常住城市是 分机 8821。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["李工 139****3344"],"hop_count":1,"answer":"李工 139****3344","attribute":"办公城市"}}
|
||||
{"id":"realistic-noise_context-00141","group_id":"realistic-noise_context-00141","source":"realistic_memory_eval","family":"noise_context","query":"我出问题该喊谁？","candidates":[{"text":"这是普通对话噪声：验证用户-016730 暂时提到一个无关编号 N-7DTA9XGG，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-859589 暂时提到一个无关编号 N-KG2GYSDW，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-402371 暂时提到一个无关编号 N-R78JN457，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-377778 暂时提到一个无关编号 N-USN57ATJ，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-840296 暂时提到一个无关编号 N-SKGPPAZW，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-886421 暂时提到一个无关编号 N-GKPSL2MX，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-662791 暂时提到一个无关编号 N-9Z6C9TDG，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-222587 暂时提到一个无关编号 N-2T9QCMS9，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[7],"positive_index":7,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["每周三 22:00"],"hop_count":1,"answer":"每周三 22:00"}}
|
||||
{"id":"realistic-long_fact-00142","group_id":"realistic-long_fact-00142","source":"realistic_memory_eval","family":"long_fact","query":"我哪条线一破就叫？","candidates":[{"text":"关于告警阈值这件事，会上也提过一嘴，但那次的说法是64GB，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于告警阈值这件事，会上也提过一嘴，但那次的说法是B-3310，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于告警阈值这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认10.20.3.7:5432，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""},{"text":"关于告警阈值这件事，会上也提过一嘴，但那次的说法是李工 139****3344，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["10.20.3.7:5432"],"hop_count":1,"answer":"10.20.3.7:5432"}}
|
||||
{"id":"realistic-unknown_attribute-00143","group_id":"realistic-unknown_attribute-00143","source":"realistic_memory_eval","family":"unknown_attribute","query":"我数据落在哪一个地址？","candidates":[{"text":"我的告警阈值是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00144","group_id":"realistic-multi_entity-00144","source":"realistic_memory_eval","family":"multi_entity","query":"李工内存开到多大？","candidates":[{"text":"张三缓存开到 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"项目 Alpha缓存开到 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"李工的缓存容量是 夜间 02:00。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["夜间 02:00"],"hop_count":1,"answer":"夜间 02:00","attribute":"缓存容量"}}
|
||||
{"id":"realistic-alias_paraphrase-00145","group_id":"realistic-alias_paraphrase-00145","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我那个数字是多少？","candidates":[{"text":"我的告警阈值是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 v2.8.0。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我每天睡醒最早看的那个数字是 128GB。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[4],"positive_index":4,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["128GB"],"hop_count":1,"answer":"128GB"}}
|
||||
{"id":"realistic-update_conflict-00146","group_id":"realistic-update_conflict-00146","source":"realistic_memory_eval","family":"update_conflict","query":"我数据落在哪一个地址？","candidates":[{"text":"我的主库地址是 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我主库落在 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3],"positive_index":3,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["B3-204"],"hop_count":1,"answer":"B3-204","attribute":"主库地址","superseded_value":"10.20.9.1:6379"}}
|
||||
{"id":"realistic-multi_hop-00147","group_id":"realistic-multi_hop-00147","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-J368D8 对应的值班人是 值班人-8038。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-U4ET7W 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-6XULGF 对应的值班人是 值班人-8189。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-5Z2GFR 对应的值班人是 值班人-4772。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-U4ET7W。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1,4],"positive_index":1,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00148","group_id":"realistic-near_miss-00148","source":"realistic_memory_eval","family":"near_miss","query":"我家座机最后几位数字是啥？","candidates":[{"text":"我的告警阈值是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的宿舍楼层是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的工位楼层是 64GB。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["每周三 22:00"],"hop_count":1,"answer":"每周三 22:00","attribute":"宿舍楼层"}}
|
||||
{"id":"realistic-noise_context-00149","group_id":"realistic-noise_context-00149","source":"realistic_memory_eval","family":"noise_context","query":"我缓冲配了多少？","candidates":[{"text":"这是普通对话噪声：验证用户-475073 暂时提到一个无关编号 N-ETHMYCQR，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 64GB。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-226673 暂时提到一个无关编号 N-B5FH8UCL，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-124908 暂时提到一个无关编号 N-SC9XDVPB，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-026199 暂时提到一个无关编号 N-U86SVZT9，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-097674 暂时提到一个无关编号 N-UHWH7QZ3，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-109561 暂时提到一个无关编号 N-E2KVHSZQ，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-677341 暂时提到一个无关编号 N-ZNLUQYA8，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-062072 暂时提到一个无关编号 N-3B7Y5JK6，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["64GB"],"hop_count":1,"answer":"64GB"}}
|
||||
{"id":"realistic-long_fact-00150","group_id":"realistic-long_fact-00150","source":"realistic_memory_eval","family":"long_fact","query":"我数据落在哪一个地址？","candidates":[{"text":"关于主库地址这件事，会上也提过一嘴，但那次的说法是王工 138****6621，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于主库地址这件事，会上也提过一嘴，但那次的说法是10.20.9.1:6379，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于主库地址这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认分机 8821，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""},{"text":"关于主库地址这件事，会上也提过一嘴，但那次的说法是v2.8.0，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["分机 8821"],"hop_count":1,"answer":"分机 8821"}}
|
||||
{"id":"realistic-unknown_attribute-00151","group_id":"realistic-unknown_attribute-00151","source":"realistic_memory_eval","family":"unknown_attribute","query":"我哪条线一破就叫？","candidates":[{"text":"我的主库地址是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 64GB。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 64GB。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00152","group_id":"realistic-multi_entity-00152","source":"realistic_memory_eval","family":"multi_entity","query":"李工到多少会触发报警？","candidates":[{"text":"我超过 A-7719 就报警。","kind":"fact","entity":"user","attribute":""},{"text":"李工的告警阈值是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"客户 Beta超过 每周三 22:00 就报警。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["B-3310"],"hop_count":1,"answer":"B-3310","attribute":"告警阈值"}}
|
||||
{"id":"realistic-alias_paraphrase-00153","group_id":"realistic-alias_paraphrase-00153","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我用的那台是什么款？","candidates":[{"text":"我对接的接口版本号是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我手上那台机器是 64GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 B3-204。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3],"positive_index":3,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["64GB"],"hop_count":1,"answer":"64GB"}}
|
||||
{"id":"realistic-update_conflict-00154","group_id":"realistic-update_conflict-00154","source":"realistic_memory_eval","family":"update_conflict","query":"我写库连的是哪里？","candidates":[{"text":"我的主库地址是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 64GB。","kind":"fact","entity":"user","attribute":""},{"text":"我主库落在 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["128GB"],"hop_count":1,"answer":"128GB","attribute":"主库地址","superseded_value":"10.20.3.7:5432"}}
|
||||
{"id":"realistic-multi_hop-00155","group_id":"realistic-multi_hop-00155","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-TJC3JV 对应的值班人是 值班人-4342。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-ALL79F 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-ALL79F。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-DV29XX 对应的值班人是 值班人-7114。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-QP975Y 对应的值班人是 值班人-5229。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1,2],"positive_index":1,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00156","group_id":"realistic-near_miss-00156","source":"realistic_memory_eval","family":"near_miss","query":"我家座机最后几位数字是啥？","candidates":[{"text":"我的座机尾号是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我的手机尾号是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 分机 8821。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["78%"],"hop_count":1,"answer":"78%","attribute":"座机尾号"}}
|
||||
{"id":"realistic-noise_context-00157","group_id":"realistic-noise_context-00157","source":"realistic_memory_eval","family":"noise_context","query":"我出问题该喊谁？","candidates":[{"text":"这是普通对话噪声：验证用户-904457 暂时提到一个无关编号 N-GBPVX6YB，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-824696 暂时提到一个无关编号 N-GW7KSL8Q，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-801615 暂时提到一个无关编号 N-V2ZZXP5V，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-182395 暂时提到一个无关编号 N-KGMTA8B2，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-546081 暂时提到一个无关编号 N-XNX53TDE，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-905952 暂时提到一个无关编号 N-CLJJUXR2，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-593340 暂时提到一个无关编号 N-3BQUX857，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-381676 暂时提到一个无关编号 N-FWHKJFRQ，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 李工 139****3344。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[8],"positive_index":8,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["李工 139****3344"],"hop_count":1,"answer":"李工 139****3344"}}
|
||||
{"id":"realistic-long_fact-00158","group_id":"realistic-long_fact-00158","source":"realistic_memory_eval","family":"long_fact","query":"我紧急情况下找谁接手？","candidates":[{"text":"关于应急联系人这件事，会上也提过一嘴，但那次的说法是v2.8.0，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于应急联系人这件事，会上也提过一嘴，但那次的说法是91.5%，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于应急联系人这件事，会上也提过一嘴，但那次的说法是A-7719，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于应急联系人这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认分机 8821，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3],"positive_index":3,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["分机 8821"],"hop_count":1,"answer":"分机 8821"}}
|
||||
{"id":"realistic-unknown_attribute-00159","group_id":"realistic-unknown_attribute-00159","source":"realistic_memory_eval","family":"unknown_attribute","query":"我出问题该喊谁？","candidates":[{"text":"我常驻机房在 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 分机 8821。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00160","group_id":"realistic-multi_entity-00160","source":"realistic_memory_eval","family":"multi_entity","query":"李工出问题该喊谁？","candidates":[{"text":"李工的应急联系人是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"客户 Beta出事找 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我出事找 v3.14.2。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["10.20.3.7:5432"],"hop_count":1,"answer":"10.20.3.7:5432","attribute":"应急联系人"}}
|
||||
{"id":"realistic-alias_paraphrase-00161","group_id":"realistic-alias_paraphrase-00161","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我那个数字是多少？","candidates":[{"text":"我的告警阈值是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我每天睡醒最早看的那个数字是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3],"positive_index":3,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["v3.14.2"],"hop_count":1,"answer":"v3.14.2"}}
|
||||
{"id":"realistic-update_conflict-00162","group_id":"realistic-update_conflict-00162","source":"realistic_memory_eval","family":"update_conflict","query":"我数据落在哪一个地址？","candidates":[{"text":"我主库落在 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 每周三 22:00。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["A-7719"],"hop_count":1,"answer":"A-7719","attribute":"主库地址","superseded_value":"91.5%"}}
|
||||
{"id":"realistic-multi_hop-00163","group_id":"realistic-multi_hop-00163","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-WKNXA4 对应的值班人是 值班人-7791。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-4RRAPK 对应的值班人是 值班人-1278。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-6UE4ZY。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-ZBEJDL 对应的值班人是 值班人-9609。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-6UE4ZY 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2,4],"positive_index":2,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00164","group_id":"realistic-near_miss-00164","source":"realistic_memory_eval","family":"near_miss","query":"我随身那台设备的末几位数字是啥？","candidates":[{"text":"我的接口版本号是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我的办公城市是 64GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的常住城市是 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 B-3310。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["64GB"],"hop_count":1,"answer":"64GB","attribute":"办公城市"}}
|
||||
{"id":"realistic-noise_context-00165","group_id":"realistic-noise_context-00165","source":"realistic_memory_eval","family":"noise_context","query":"我有急事按哪个号找人？","candidates":[{"text":"我的值班电话是 v2.8.0。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-857231 暂时提到一个无关编号 N-5MQXHYUS，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-745830 暂时提到一个无关编号 N-9WZZHRDJ，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-300032 暂时提到一个无关编号 N-9ESHHDSU，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-596482 暂时提到一个无关编号 N-MB66EMD9，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-160421 暂时提到一个无关编号 N-7LF3U35F，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-105384 暂时提到一个无关编号 N-Q4Z8H6AN，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-630853 暂时提到一个无关编号 N-XY2Z98FU，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-366851 暂时提到一个无关编号 N-LAU2PQH6，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["v2.8.0"],"hop_count":1,"answer":"v2.8.0"}}
|
||||
{"id":"realistic-long_fact-00166","group_id":"realistic-long_fact-00166","source":"realistic_memory_eval","family":"long_fact","query":"我出问题该喊谁？","candidates":[{"text":"关于应急联系人这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认李工 139****3344，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""},{"text":"关于应急联系人这件事，会上也提过一嘴，但那次的说法是10.20.9.1:6379，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于应急联系人这件事，会上也提过一嘴，但那次的说法是78%，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于应急联系人这件事，会上也提过一嘴，但那次的说法是10.20.9.1:6379，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["李工 139****3344"],"hop_count":1,"answer":"李工 139****3344"}}
|
||||
{"id":"realistic-unknown_attribute-00167","group_id":"realistic-unknown_attribute-00167","source":"realistic_memory_eval","family":"unknown_attribute","query":"我数据落在哪一个地址？","candidates":[{"text":"我的告警阈值是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 v3.14.2。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00168","group_id":"realistic-multi_entity-00168","source":"realistic_memory_eval","family":"multi_entity","query":"Beta 客户一般什么时候做上线？","candidates":[{"text":"张三固定 B3-204 做上线。","kind":"fact","entity":"user","attribute":""},{"text":"客户 Beta的发布窗口是 64GB。","kind":"fact","entity":"user","attribute":""},{"text":"李工固定 10.20.9.1:6379 做上线。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["64GB"],"hop_count":1,"answer":"64GB","attribute":"发布窗口"}}
|
||||
{"id":"realistic-alias_paraphrase-00169","group_id":"realistic-alias_paraphrase-00169","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我那个数字是多少？","candidates":[{"text":"我常驻机房在 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我每天睡醒最早看的那个数字是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["夜间 02:00"],"hop_count":1,"answer":"夜间 02:00"}}
|
||||
{"id":"realistic-update_conflict-00170","group_id":"realistic-update_conflict-00170","source":"realistic_memory_eval","family":"update_conflict","query":"我联调用的那个版本是什么？","candidates":[{"text":"我目前跑的是 每周三 22:00 版本接口。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 王工 138****6621。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 分机 8821。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["每周三 22:00"],"hop_count":1,"answer":"每周三 22:00","attribute":"接口版本号","superseded_value":"128GB"}}
|
||||
{"id":"realistic-multi_hop-00171","group_id":"realistic-multi_hop-00171","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"张三负责的那个项目的内部编号是 REF-NF238X。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-NF238X 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-TS7J46 对应的值班人是 值班人-8724。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-6DBK9J 对应的值班人是 值班人-5115。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-5MFY24 对应的值班人是 值班人-1901。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0,1],"positive_index":0,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00172","group_id":"realistic-near_miss-00172","source":"realistic_memory_eval","family":"near_miss","query":"我随身那台设备的末几位数字是啥？","candidates":[{"text":"我的常驻机房是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我的办公城市是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的常住城市是 分机 8821。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["10.20.3.7:5432"],"hop_count":1,"answer":"10.20.3.7:5432","attribute":"办公城市"}}
|
||||
{"id":"realistic-noise_context-00173","group_id":"realistic-noise_context-00173","source":"realistic_memory_eval","family":"noise_context","query":"我轮班时该拨什么号码？","candidates":[{"text":"这是普通对话噪声：验证用户-095100 暂时提到一个无关编号 N-C9BMLFSF，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-952039 暂时提到一个无关编号 N-GR9LHBYK，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-176845 暂时提到一个无关编号 N-UPXH5QV3，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-982839 暂时提到一个无关编号 N-LGFB4BSV，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-555718 暂时提到一个无关编号 N-TQUKLGDM，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-099138 暂时提到一个无关编号 N-3FFWC95R，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-330540 暂时提到一个无关编号 N-D4EUPH3A，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-473002 暂时提到一个无关编号 N-TFTTU4NT，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["A-7719"],"hop_count":1,"answer":"A-7719"}}
|
||||
{"id":"realistic-long_fact-00174","group_id":"realistic-long_fact-00174","source":"realistic_memory_eval","family":"long_fact","query":"我出问题该喊谁？","candidates":[{"text":"关于应急联系人这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认v3.14.2，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""},{"text":"关于应急联系人这件事，会上也提过一嘴，但那次的说法是128GB，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于应急联系人这件事，会上也提过一嘴，但那次的说法是10.20.9.1:6379，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于应急联系人这件事，会上也提过一嘴，但那次的说法是B-3310，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0],"positive_index":0,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["v3.14.2"],"hop_count":1,"answer":"v3.14.2"}}
|
||||
{"id":"realistic-unknown_attribute-00175","group_id":"realistic-unknown_attribute-00175","source":"realistic_memory_eval","family":"unknown_attribute","query":"我出问题该喊谁？","candidates":[{"text":"我的主库地址是 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 每周三 22:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 A-7719。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00176","group_id":"realistic-multi_entity-00176","source":"realistic_memory_eval","family":"multi_entity","query":"李工一般什么时候做上线？","candidates":[{"text":"我固定 10.20.9.1:6379 做上线。","kind":"fact","entity":"user","attribute":""},{"text":"李工的发布窗口是 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"张三固定 128GB 做上线。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["B3-204"],"hop_count":1,"answer":"B3-204","attribute":"发布窗口"}}
|
||||
{"id":"realistic-alias_paraphrase-00177","group_id":"realistic-alias_paraphrase-00177","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我常年落脚在哪儿？","candidates":[{"text":"我对接的接口版本号是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我平时待得最久的那座城是 v2.8.0。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[4],"positive_index":4,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["v2.8.0"],"hop_count":1,"answer":"v2.8.0"}}
|
||||
{"id":"realistic-update_conflict-00178","group_id":"realistic-update_conflict-00178","source":"realistic_memory_eval","family":"update_conflict","query":"我平时在哪间屋子干活？","candidates":[{"text":"我常驻机房在 v2.8.0。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 B3-204。","kind":"fact","entity":"user","attribute":""},{"text":"我平时待的机房是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 王工 138****6621。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3],"positive_index":3,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["10.20.9.1:6379"],"hop_count":1,"answer":"10.20.9.1:6379","attribute":"常驻机房","superseded_value":"78%"}}
|
||||
{"id":"realistic-multi_hop-00179","group_id":"realistic-multi_hop-00179","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-UATAGF 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-D4HAYH 对应的值班人是 值班人-6789。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-4HNT4M 对应的值班人是 值班人-7157。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-UATAGF。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-JVKSAZ 对应的值班人是 值班人-6351。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0,3],"positive_index":0,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00180","group_id":"realistic-near_miss-00180","source":"realistic_memory_eval","family":"near_miss","query":"我随身那台设备的末几位数字是啥？","candidates":[{"text":"我的应急联系人是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的宿舍楼层是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的工位楼层是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 91.5%。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["10.20.9.1:6379"],"hop_count":1,"answer":"10.20.9.1:6379","attribute":"工位楼层"}}
|
||||
{"id":"realistic-noise_context-00181","group_id":"realistic-noise_context-00181","source":"realistic_memory_eval","family":"noise_context","query":"我平时在哪间屋子干活？","candidates":[{"text":"这是普通对话噪声：验证用户-462352 暂时提到一个无关编号 N-J9M3XYGA，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-382311 暂时提到一个无关编号 N-UAT7D5WY，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-005141 暂时提到一个无关编号 N-SMBEPE4V，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-929339 暂时提到一个无关编号 N-MU2RR35B，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-095614 暂时提到一个无关编号 N-GE7M2K9J，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-243954 暂时提到一个无关编号 N-JADFJ86F，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-700295 暂时提到一个无关编号 N-J34RGLZZ，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-291729 暂时提到一个无关编号 N-KXEJ3ZKX，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[4],"positive_index":4,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["128GB"],"hop_count":1,"answer":"128GB"}}
|
||||
{"id":"realistic-long_fact-00182","group_id":"realistic-long_fact-00182","source":"realistic_memory_eval","family":"long_fact","query":"我写库连的是哪里？","candidates":[{"text":"关于主库地址这件事，会上也提过一嘴，但那次的说法是10.20.3.7:5432，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于主库地址这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认91.5%，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""},{"text":"关于主库地址这件事，会上也提过一嘴，但那次的说法是每周三 22:00，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于主库地址这件事，会上也提过一嘴，但那次的说法是128GB，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["91.5%"],"hop_count":1,"answer":"91.5%"}}
|
||||
{"id":"realistic-unknown_attribute-00183","group_id":"realistic-unknown_attribute-00183","source":"realistic_memory_eval","family":"unknown_attribute","query":"我数据落在哪一个地址？","candidates":[{"text":"我对接的接口版本号是 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 128GB。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00184","group_id":"realistic-multi_entity-00184","source":"realistic_memory_eval","family":"multi_entity","query":"Beta 客户哪条线一破就叫？","candidates":[{"text":"项目 Alpha超过 10.20.3.7:5432 就报警。","kind":"fact","entity":"user","attribute":""},{"text":"客户 Beta的告警阈值是 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"张三超过 王工 138****6621 就报警。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["分机 8821"],"hop_count":1,"answer":"分机 8821","attribute":"告警阈值"}}
|
||||
{"id":"realistic-alias_paraphrase-00185","group_id":"realistic-alias_paraphrase-00185","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我常年落脚在哪儿？","candidates":[{"text":"我的发布窗口是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我常驻机房在 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"我平时待得最久的那座城是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 王工 138****6621。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[3],"positive_index":3,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["78%"],"hop_count":1,"answer":"78%"}}
|
||||
{"id":"realistic-update_conflict-00186","group_id":"realistic-update_conflict-00186","source":"realistic_memory_eval","family":"update_conflict","query":"我缓冲配了多少？","candidates":[{"text":"我的缓存容量是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 10.20.9.1:6379。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 v2.8.0。","kind":"fact","entity":"user","attribute":""},{"text":"我缓存开到 91.5%。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[4],"positive_index":4,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["91.5%"],"hop_count":1,"answer":"91.5%","attribute":"缓存容量","superseded_value":"10.20.3.7:5432"}}
|
||||
{"id":"realistic-multi_hop-00187","group_id":"realistic-multi_hop-00187","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-W8QLKU 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-5M59J3 对应的值班人是 值班人-5519。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-QBCRBM 对应的值班人是 值班人-8405。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-T8L9FB 对应的值班人是 值班人-5717。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-W8QLKU。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[0,4],"positive_index":0,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00188","group_id":"realistic-near_miss-00188","source":"realistic_memory_eval","family":"near_miss","query":"我家座机最后几位数字是啥？","candidates":[{"text":"我的办公城市是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我的接口版本号是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的常住城市是 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"我的常驻机房是 91.5%。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["李工 139****3344"],"hop_count":1,"answer":"李工 139****3344","attribute":"常住城市"}}
|
||||
{"id":"realistic-noise_context-00189","group_id":"realistic-noise_context-00189","source":"realistic_memory_eval","family":"noise_context","query":"我紧急情况下找谁接手？","candidates":[{"text":"这是普通对话噪声：验证用户-197412 暂时提到一个无关编号 N-EWSUNUEX，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-501624 暂时提到一个无关编号 N-2AS8CAMD，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-129219 暂时提到一个无关编号 N-GT2F97EV，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-322515 暂时提到一个无关编号 N-V64T2XVP，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-747279 暂时提到一个无关编号 N-Q8J82BNA，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-271169 暂时提到一个无关编号 N-J3B3B3AW，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-721152 暂时提到一个无关编号 N-B5CN2494，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-219188 暂时提到一个无关编号 N-KE5BNU3S，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["李工 139****3344"],"hop_count":1,"answer":"李工 139****3344"}}
|
||||
{"id":"realistic-long_fact-00190","group_id":"realistic-long_fact-00190","source":"realistic_memory_eval","family":"long_fact","query":"我内存开到多大？","candidates":[{"text":"关于缓存容量这件事，会上也提过一嘴，但那次的说法是B3-204，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于缓存容量这件事，会上也提过一嘴，但那次的说法是B-3310，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于缓存容量这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认A-7719，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""},{"text":"关于缓存容量这件事，会上也提过一嘴，但那次的说法是v3.14.2，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["A-7719"],"hop_count":1,"answer":"A-7719"}}
|
||||
{"id":"realistic-unknown_attribute-00191","group_id":"realistic-unknown_attribute-00191","source":"realistic_memory_eval","family":"unknown_attribute","query":"我一般什么时候做上线？","candidates":[{"text":"我的值班电话是 李工 139****3344。","kind":"fact","entity":"user","attribute":""},{"text":"我的主库地址是 64GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的缓存容量是 78%。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 128GB。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
{"id":"realistic-multi_entity-00192","group_id":"realistic-multi_entity-00192","source":"realistic_memory_eval","family":"multi_entity","query":"运维那位李工固定在哪个位置办公？","candidates":[{"text":"张三平时待的机房是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"李工常驻机房在 分机 8821。","kind":"fact","entity":"user","attribute":""},{"text":"客户 Beta平时待的机房是 v2.8.0。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"multi_entity","answerable":true,"acceptable":["分机 8821"],"hop_count":1,"answer":"分机 8821","attribute":"常驻机房"}}
|
||||
{"id":"realistic-alias_paraphrase-00193","group_id":"realistic-alias_paraphrase-00193","source":"realistic_memory_eval","family":"alias_paraphrase","query":"我用的那台是什么款？","candidates":[{"text":"我常驻机房在 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我手上那台机器是 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的告警阈值是 64GB。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 李工 139****3344。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"alias_paraphrase","answerable":true,"acceptable":["91.5%"],"hop_count":1,"answer":"91.5%"}}
|
||||
{"id":"realistic-update_conflict-00194","group_id":"realistic-update_conflict-00194","source":"realistic_memory_eval","family":"update_conflict","query":"我有急事按哪个号找人？","candidates":[{"text":"我的值班电话是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我留的值班号码为 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 B-3310。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 91.5%。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 李工 139****3344。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"update_conflict","answerable":true,"acceptable":["91.5%"],"hop_count":1,"answer":"91.5%","attribute":"值班电话","superseded_value":"128GB"}}
|
||||
{"id":"realistic-multi_hop-00195","group_id":"realistic-multi_hop-00195","source":"realistic_memory_eval","family":"multi_hop","query":"负责那个项目的同事对应的值班人是谁？","candidates":[{"text":"编号 REF-RX9LFR 对应的值班人是 值班人-4728。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-V6H3BU 对应的值班人是 值班人-5935。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-QHN8Z7 对应的值班人是 值班人-7741。","kind":"fact","entity":"user","attribute":""},{"text":"编号 REF-JH44V5 对应的值班人是 值班人-3816。","kind":"fact","entity":"user","attribute":""},{"text":"张三负责的那个项目的内部编号是 REF-QHN8Z7。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2,4],"positive_index":2,"need_memory":1.0,"hop":2,"metadata":{"category":"multi_hop","answerable":true,"acceptable":["值班人-7741"],"hop_count":2,"answer":"值班人-7741"}}
|
||||
{"id":"realistic-near_miss-00196","group_id":"realistic-near_miss-00196","source":"realistic_memory_eval","family":"near_miss","query":"我家座机最后几位数字是啥？","candidates":[{"text":"我的应急联系人是 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 A-7719。","kind":"fact","entity":"user","attribute":""},{"text":"我的座机尾号是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我的手机尾号是 A-7719。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[2],"positive_index":2,"need_memory":1.0,"hop":1,"metadata":{"category":"near_miss","answerable":true,"acceptable":["v3.14.2"],"hop_count":1,"answer":"v3.14.2","attribute":"座机尾号"}}
|
||||
{"id":"realistic-noise_context-00197","group_id":"realistic-noise_context-00197","source":"realistic_memory_eval","family":"noise_context","query":"我挑哪个时段变更？","candidates":[{"text":"这是普通对话噪声：验证用户-876800 暂时提到一个无关编号 N-6R2NC6U4，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-466934 暂时提到一个无关编号 N-MHBNWL35，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-199079 暂时提到一个无关编号 N-UZZ6JEKM，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-368162 暂时提到一个无关编号 N-6CXNRTAS，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-493619 暂时提到一个无关编号 N-PSRL3CG7，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-796745 暂时提到一个无关编号 N-3LZN5HPN，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-118841 暂时提到一个无关编号 N-UT8VZESA，不需要长期保存。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"这是普通对话噪声：验证用户-105474 暂时提到一个无关编号 N-6KL2FSUN，不需要长期保存。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[7],"positive_index":7,"need_memory":1.0,"hop":1,"metadata":{"category":"noise_context","answerable":true,"acceptable":["128GB"],"hop_count":1,"answer":"128GB"}}
|
||||
{"id":"realistic-long_fact-00198","group_id":"realistic-long_fact-00198","source":"realistic_memory_eval","family":"long_fact","query":"我写库连的是哪里？","candidates":[{"text":"关于主库地址这件事，会上也提过一嘴，但那次的说法是v3.14.2，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于主库地址这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认v3.14.2，后续如果还有调整会在群里同步，暂时先按这个执行。","kind":"fact","entity":"user","attribute":""},{"text":"关于主库地址这件事，会上也提过一嘴，但那次的说法是A-7719，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""},{"text":"关于主库地址这件事，会上也提过一嘴，但那次的说法是64GB，后来没有正式确认过。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[1],"positive_index":1,"need_memory":1.0,"hop":1,"metadata":{"category":"long_fact","answerable":true,"acceptable":["v3.14.2"],"hop_count":1,"answer":"v3.14.2"}}
|
||||
{"id":"realistic-unknown_attribute-00199","group_id":"realistic-unknown_attribute-00199","source":"realistic_memory_eval","family":"unknown_attribute","query":"我哪条线一破就叫？","candidates":[{"text":"我常驻机房在 10.20.3.7:5432。","kind":"fact","entity":"user","attribute":""},{"text":"我的值班电话是 v3.14.2。","kind":"fact","entity":"user","attribute":""},{"text":"我的应急联系人是 夜间 02:00。","kind":"fact","entity":"user","attribute":""},{"text":"我的发布窗口是 128GB。","kind":"fact","entity":"user","attribute":""},{"text":"我对接的接口版本号是 128GB。","kind":"fact","entity":"user","attribute":""}],"positive_indices":[],"positive_index":-1,"need_memory":0.0,"hop":0,"metadata":{"category":"unknown_attribute","answerable":false,"acceptable":[],"hop_count":0,"answer":""}}
|
||||
Reference in New Issue
Block a user