wpy/natural-memory-nm21
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
natural-memory-nm21/rv2_topk2_e2e.json
T
WpyQwq 643e22ecb9 Natural Memory NM2.1: 记忆路由器分叉、数据集缺陷修复与全轴评测证据 
- 引入 MemoryRouterXL 与 v5/v6 流式多线程训练/编码管线
- 修复 prepare_memory_router_dataset 候选池重建缺陷（mega 家族 3568x 加速，输出逐字节相同）
- 修复 v5 被破坏的拒答与多跳标签（train 未知样本 319 -> 16319，multi_hop 平均正例 1.00 -> 2.00）
- 同存储预算下 V2-128 v6 逐轴 22/22 通过：Top-1 41.12% -> 94.62%，未知拒答 0.00% -> 100.00%
- 记录三条被实测推翻的显然优化（logits_to_keep=1 反而慢 55%、XL 容量未带来收益）
- 记忆手术跨架构可移植性 14/14，读写关闭时与原生模型逐位相同
2026-09-19 11:11:31 +08:00

2315 lines
74 KiB
JSON
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
{
"realistic_v2": {
"summary": {
"router": "realistic_v2",
"cases": 200,
"accuracy_pct": 58.0,
"answerable_cases": 175,
"answerable_accuracy_pct": 66.28571428571429,
"unknown_cases": 25,
"unknown_refusal_pct": 0.0,
"wrong_abstention_pct": 2.2857142857142856,
"seconds": 489.2,
"per_category": {
"alias_paraphrase": {
"cases": 25,
"accuracy_pct": 92.0,
"answerable": 25,
"answerable_accuracy_pct": 92.0,
"unknown": 0,
"unknown_refusal_pct": 0.0
},
"long_fact": {
"cases": 25,
"accuracy_pct": 48.0,
"answerable": 25,
"answerable_accuracy_pct": 48.0,
"unknown": 0,
"unknown_refusal_pct": 0.0
},
"multi_entity": {
"cases": 25,
"accuracy_pct": 92.0,
"answerable": 25,
"answerable_accuracy_pct": 92.0,
"unknown": 0,
"unknown_refusal_pct": 0.0
},
"multi_hop": {
"cases": 25,
"accuracy_pct": 36.0,
"answerable": 25,
"answerable_accuracy_pct": 36.0,
"unknown": 0,
"unknown_refusal_pct": 0.0
},
"near_miss": {
"cases": 25,
"accuracy_pct": 80.0,
"answerable": 25,
"answerable_accuracy_pct": 80.0,
"unknown": 0,
"unknown_refusal_pct": 0.0
},
"noise_context": {
"cases": 25,
"accuracy_pct": 72.0,
"answerable": 25,
"answerable_accuracy_pct": 72.0,
"unknown": 0,
"unknown_refusal_pct": 0.0
},
"unknown_attribute": {
"cases": 25,
"accuracy_pct": 0.0,
"answerable": 0,
"answerable_accuracy_pct": 0.0,
"unknown": 25,
"unknown_refusal_pct": 0.0
},
"update_conflict": {
"cases": 25,
"accuracy_pct": 44.0,
"answerable": 25,
"answerable_accuracy_pct": 44.0,
"unknown": 0,
"unknown_refusal_pct": 0.0
}
}
},
"rows": [
{
"category": "alias_paraphrase",
"query": "我用的那台是什么款？",
"reply": "我手上那台机器是 v2.8.0。",
"written": 5,
"correct": true,
"matched": [
"v2.8.0"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我常年落脚在哪儿？",
"reply": "我常年落脚在凌晨 03:30。",
"written": 5,
"correct": true,
"matched": [
"凌晨 03:30"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我常去的那家叫什么？",
"reply": "我周末常去的那家店叫 C1-105。",
"written": 5,
"correct": true,
"matched": [
"C1-105"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我用的那台是什么款？",
"reply": "我手上那台机器是 A-7719。",
"written": 5,
"correct": true,
"matched": [
"A-7719"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我那个数字是多少？",
"reply": "您的仓库库位是 64GB，您的值班表是 钱工 135****2211。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我常去的那家叫什么？",
"reply": "我周末常去的那家店叫 C1-105。",
"written": 5,
"correct": true,
"matched": [
"C1-105"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我常去的那家叫什么？",
"reply": "我周末常去的那家店叫 每周一 09:30。",
"written": 5,
"correct": true,
"matched": [
"每周一 09:30"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我常年落脚在哪儿？",
"reply": "我常年落脚在 256GB。",
"written": 5,
"correct": true,
"matched": [
"256GB"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我那个数字是多少？",
"reply": "我那个数字是 B-3310。",
"written": 5,
"correct": true,
"matched": [
"B-3310"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我随身带的是哪一本？",
"reply": "我包里常备的那本册子是 10.20.3.7:5432。",
"written": 5,
"correct": true,
"matched": [
"10.20.3.7:5432"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我常去的那家叫什么？",
"reply": "我周末常去的那家店叫 10.30.7.3:6379。",
"written": 5,
"correct": true,
"matched": [
"10.30.7.3:6379"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我早上那杯是什么口味？",
"reply": "我每天早上灌下去的那杯是偏爱 v2.8.0 口味。",
"written": 5,
"correct": true,
"matched": [
"v2.8.0"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我用的那台是什么款？",
"reply": "我手上那台机器是 63.2%。",
"written": 5,
"correct": true,
"matched": [
"63.2%"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我早上那杯是什么口味？",
"reply": "我每天早上灌下去的那杯是偏爱 v1.9.7 口味。",
"written": 5,
"correct": true,
"matched": [
"v1.9.7"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我常年落脚在哪儿？",
"reply": "我平时待得最久的那座城是 32GB。",
"written": 5,
"correct": true,
"matched": [
"32GB"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我那个数字是多少？",
"reply": "我每天睡醒最早看的那个数字是 C1-105。",
"written": 5,
"correct": true,
"matched": [
"C1-105"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我早上那杯是什么口味？",
"reply": "我每天早上灌下去的那杯偏爱 B-3310 的口味信息在现有记录中未明确说明，无法确认具体口味。",
"written": 5,
"correct": true,
"matched": [
"B-3310"
],
"abstained": false,
"wrongly_abstained": true
},
{
"category": "alias_paraphrase",
"query": "我随身带的是哪一本？",
"reply": "我包里常备的那本册子是 10.20.9.1:6379。",
"written": 5,
"correct": true,
"matched": [
"10.20.9.1:6379"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我常去的那家叫什么？",
"reply": "我周末常去的那家店叫 v1.9.7。",
"written": 5,
"correct": true,
"matched": [
"v1.9.7"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我随身带的是哪一本？",
"reply": "我随身带的是 v2.8.0。",
"written": 5,
"correct": true,
"matched": [
"v2.8.0"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我用的那台是什么款？",
"reply": "我手上那台机器是 每周三 22:00。",
"written": 5,
"correct": true,
"matched": [
"每周三 22:00"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我早上那杯是什么口味？",
"reply": "我每天早上灌下去的那杯是偏爱 v4.0.0-rc1 口味。",
"written": 5,
"correct": true,
"matched": [
"v4.0.0-rc1"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我那个数字是多少？",
"reply": "您的保险到期数字是 李工 139****3344。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我用的那台是什么款？",
"reply": "我手上那台机器是 v2.8.0。",
"written": 5,
"correct": true,
"matched": [
"v2.8.0"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "alias_paraphrase",
"query": "我早上那杯是什么口味？",
"reply": "我每天早上灌下去的那杯偏爱 凌晨 03:30 的口味是未知，因为长期记忆中仅记录了时间（凌晨 03:30）和饮品名称（偏爱），未包含具体的口味信息。",
"written": 5,
"correct": true,
"matched": [
"凌晨 03:30"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我协议编号是什么？",
"reply": "关于合同编号，会上曾提及的说法是10.20.9.1:6379，但后来没有正式确认过；散会时临时讨论后确认的临时值是128GB，后续如有",
"written": 4,
"correct": true,
"matched": [
"128GB"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我轮班时该拨什么号码？",
"reply": "关于值班电话，之前散会时临时定下来的是85%，后续如果还有调整会在群里同步，暂时先按这个执行。",
"written": 4,
"correct": true,
"matched": [
"85%"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我挑哪个时段变更？",
"reply": "关于发布窗口的具体时段，目前没有正式确认过，会上曾提及的说法是B-3310。",
"written": 4,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我上游是哪一家？",
"reply": "关于供应商这件事，会上也提过一嘴，但那次的说法是A-7719，后来没有正式确认过。",
"written": 4,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我上手需要多长时间？",
"reply": "关于培训周期这件事，会上也提过一嘴，但那次的说法是v3.14.2，后来没有正式确认过。",
"written": 4,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我这周按什么表轮？",
"reply": "关于值班表，之前散会时临时定下来按 A-7719 轮，后续如有调整会在群里同步。",
"written": 4,
"correct": true,
"matched": [
"A-7719"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我纸面上那个号是多少？",
"reply": "关于合同编号这件事，会上也提过一嘴，但那次的说法是10.30.1.9:5432，后来没有正式确认过。",
"written": 4,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我纸面上那个号是多少？",
"reply": "关于合同编号这件事，会上也提过一嘴，但那次的说法是91.5%，后来没有正式确认过。",
"written": 4,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我要学多久？",
"reply": "关于培训周期，之前散会时临时定了下来，当时讨论得比较久，最后确认10.20.3.7:5432，后续如果还有调整会在群里同步，暂时先按这个执行",
"written": 4,
"correct": true,
"matched": [
"10.20.3.7:5432"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我要学多久？",
"reply": "关于培训周期，之前散会时我们临时定了下来，当时讨论得比较久，最后确认A-7719，后续如果还有调整会在群里同步，暂时先按这个执行。",
"written": 4,
"correct": true,
"matched": [
"A-7719"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我挑哪个时段变更？",
"reply": "关于发布窗时段变更，会上曾提及的说法是32GB，但后来没有正式确认过。",
"written": 4,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我一般什么时候做上线？",
"reply": "关于发布窗口，会上曾提及的说法是 v1.9.7，但后来没有正式确认过。",
"written": 4,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我上游是哪一家？",
"reply": "关于您上游是哪一家，现有长期记忆中没有明确记录该信息。记忆仅包含关于供应商会议时间的模糊提及（会上提过一嘴，说法是每周一 09:30，后来没有正式确认过）",
"written": 4,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我轮班时该拨什么号码？",
"reply": "关于值班电话，之前散会时临时定下来的是夜间 02:00，后续如果还有调整会在群里同步，暂时先按这个执行。而会上也曾提到过 10.30.1.",
"written": 4,
"correct": true,
"matched": [
"夜间 02:00"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我团队什么时候碰一次？",
"reply": "关于周会时间，会上也提过一嘴，但那次的说法是赵工 137****8890，后来没有正式确认过。",
"written": 4,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我哪条线一破就叫？",
"reply": "关于告警阈值这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认分机 8821，后续如果还有调整会在群里同步，暂时先按这个执行。",
"written": 4,
"correct": true,
"matched": [
"分机 8821"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我这份多久后失效？",
"reply": "关于保险到期这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认v2.8.0，后续如果还有调整会在群里同步，暂时先按这个执行。",
"written": 4,
"correct": true,
"matched": [
"v2.8.0"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我 weekly 的会安排在何时？",
"reply": "周会时间已确认为 A-7719，后续如有调整会在群里同步，暂时先按这个执行。",
"written": 4,
"correct": true,
"matched": [
"A-7719"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我一般什么时候做上线？",
"reply": "关于发布窗口这件事，会上也提过一嘴，但那次的说法是B3-204，后来没有正式确认过。",
"written": 4,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我货是谁供的？",
"reply": "关于供应商这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认 91.5%，后续如果还有调整会在群里同步，暂时先按这个执行。",
"written": 4,
"correct": true,
"matched": [
"91.5%"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我排班依据是哪一份？",
"reply": "关于值班表，会上曾提及的说法是B3-204，后来没有正式确认过；关于值班表，会上也提过一嘴，但那次的说法是v4.0.0-rc1，后来",
"written": 4,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我数据落在哪一个地址？",
"reply": "关于主库地址，之前散会时临时定下来的是 v3.14.2，后续如有调整会在群里同步，暂时先按这个执行。",
"written": 4,
"correct": true,
"matched": [
"v3.14.2"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我哪条线一破就叫？",
"reply": "关于告警阈值这件事，之前散会时我们临时定了下来，当时讨论得比较久，最后确认分机 8821，后续如果还有调整会在群里同步，暂时先按这个执行。",
"written": 4,
"correct": true,
"matched": [
"分机 8821"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我上手需要多长时间？",
"reply": "关于培训周期这件事，会上也提过一嘴，但那次的说法是A-7719，后来没有正式确认过。",
"written": 4,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "long_fact",
"query": "我团队什么时候碰一次？",
"reply": "关于周会时间，会上也提过一嘴，但那次的说法是钱工 135****2211，后来没有正式确认过。",
"written": 4,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "李工排班依据是哪一份？",
"reply": "李工排班依据是每周一 09:30。",
"written": 3,
"correct": true,
"matched": [
"每周一 09:30"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "Beta 客户保障什么时候结束？",
"reply": "Beta 客户保障到期时间是 王工 138****6621。",
"written": 3,
"correct": true,
"matched": [
"王工 138****6621"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "李工上游是哪一家？",
"reply": "李工的供应商是 B-3310。",
"written": 3,
"correct": true,
"matched": [
"B-3310"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "Alpha东西放在哪一格？",
"reply": "项目 Alpha 的仓库库位是 王工 138****6621。",
"written": 3,
"correct": true,
"matched": [
"王工 138****6621"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "Gamma保障什么时候结束？",
"reply": "Gamma 保障的保险到期时间是 王工 138****6621。",
"written": 3,
"correct": true,
"matched": [
"王工 138****6621"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "Beta 客户保障什么时候结束？",
"reply": "客户 Beta 的保险到期是 D-9021。",
"written": 3,
"correct": true,
"matched": [
"D-9021"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "Beta 客户 weekly 的会安排在何时？",
"reply": "Beta 客户周会安排在 256GB。",
"written": 3,
"correct": true,
"matched": [
"256GB"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "李工轮班时该拨什么号码？",
"reply": "李工轮班时该拨打的号码是 256GB。",
"written": 3,
"correct": true,
"matched": [
"256GB"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "Beta到多少会触发报警？",
"reply": "Beta 超过 D-9021 就报警。",
"written": 3,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "张三团队什么时候碰一次？",
"reply": "张三团队的周会时间是 10.30.1.9:5432。",
"written": 3,
"correct": true,
"matched": [
"10.30.1.9:5432"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "Beta这周按什么表轮？",
"reply": "Beta 这周按 32GB 表轮。",
"written": 3,
"correct": true,
"matched": [
"32GB"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "Delta 客户有急事按哪个号找人？",
"reply": "Delta 客户有急事按 85% 这个号找人。",
"written": 3,
"correct": true,
"matched": [
"85%"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "王五写库连的是哪里？",
"reply": "王五的写库连接地址是 10.20.9.1:6379。",
"written": 3,
"correct": true,
"matched": [
"10.20.9.1:6379"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "Beta协议编号是什么？",
"reply": "Beta 协议编号是 夜间 02:00。",
"written": 3,
"correct": true,
"matched": [
"夜间 02:00"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "张三团队什么时候碰一次？",
"reply": "张三团队的周会时间是 10.20.3.7:5432。",
"written": 3,
"correct": true,
"matched": [
"10.20.3.7:5432"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "Gamma 项目有急事按哪个号找人？",
"reply": "Gamma 项目有急事按王工 138****6621 找人。",
"written": 3,
"correct": true,
"matched": [
"王工 138****6621"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "我 weekly 的会安排在何时？",
"reply": "张三固定在 10.20.3.7:5432 开周会，王五固定在 10.20.9.1:6379 开周会。",
"written": 3,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "Delta有急事按哪个号找人？",
"reply": "Delta 的值班电话是 E-4408。",
"written": 3,
"correct": true,
"matched": [
"E-4408"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "李工东西放在哪一格？",
"reply": "李工东西放在分机 8821 格。",
"written": 3,
"correct": true,
"matched": [
"分机 8821"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "Delta 客户东西放在哪一格？",
"reply": "Delta 客户东西放在 B3-204 格。",
"written": 3,
"correct": true,
"matched": [
"B3-204"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "Beta这份多久后失效？",
"reply": "客户 Beta 的保险到期是 v1.9.7。",
"written": 3,
"correct": true,
"matched": [
"v1.9.7"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "我们组那位张三到多少会触发报警？",
"reply": "张三的告警阈值是 v2.8.0。",
"written": 3,
"correct": true,
"matched": [
"v2.8.0"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "Gamma weekly 的会安排在何时？",
"reply": "项目 Gamma 的周会时间是分机 8821。",
"written": 3,
"correct": true,
"matched": [
"分机 8821"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "Delta要学多久？",
"reply": "客户 Delta 的培训周期是 64GB。",
"written": 3,
"correct": true,
"matched": [
"64GB"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_entity",
"query": "Alpha保障什么时候结束？",
"reply": "项目 Alpha 的保险到期是 E-4408。",
"written": 3,
"correct": true,
"matched": [
"E-4408"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-CA2EAT 对应的值班人是 值班人-8932，编号 REF-KCZRS5 对应的值班人是 值班人-3044。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-CPDVCR 对应的值班人是 值班人-8908，编号 REF-NVN3HY 对应的值班人是 值班人-6542。",
"written": 5,
"correct": true,
"matched": [
"值班人-6542"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-VBSJSC 对应的值班人是 值班人-3193，编号 REF-3974SH 对应的值班人是 值班人-2296。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-WBBJVS 对应的值班人是 值班人 -4198，编号 REF-8PBBFL 对应的值班人是 值班人 -4534。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-ULT8GG 对应的值班人是 值班人-5997，编号 REF-QGBYWN 对应的值班人是 值班人-8344。",
"written": 5,
"correct": true,
"matched": [
"值班人-8344"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-SE7JMG 对应的值班人是 值班人-2613；编号 REF-5GGJZ5 对应的值班人是 值班人-5725。",
"written": 5,
"correct": true,
"matched": [
"值班人-2613"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-S63RSN 对应的值班人是 值班人-3738，编号 REF-WJAXC7 对应的值班人是 值班人-7179。",
"written": 5,
"correct": true,
"matched": [
"值班人-3738"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-QPM7WY 对应的值班人是 值班人-6378，编号 REF-NWVMML 对应的值班人是 值班人-7685。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-VKXBFN 对应的值班人是 值班人-6025，编号 REF-BNHNGX 对应的值班人是 值班人-3951。",
"written": 5,
"correct": true,
"matched": [
"值班人-3951"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-JMUFY6 对应的值班人是 值班人-7819，编号 REF-7E6QN6 对应的值班人是 值班人-1547。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-G229JD 对应的值班人是 值班人-8014，编号 REF-LQPXN5 对应的值班人是 值班人-7311。",
"written": 5,
"correct": true,
"matched": [
"值班人-7311"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-BCUDK9 对应的值班人是 值班人 -4472，编号 REF-X83SS6 对应的值班人是 值班人 -5553。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-B3JUCF 对应的值班人是 值班人-5296。编号 REF-FDP9LT 对应的值班人是 值班人-2777。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-7ABRRH 对应的值班人是 值班人-2126。",
"written": 5,
"correct": true,
"matched": [
"值班人-2126"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-JFG5SQ 对应的值班人是 值班人 -5761。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-VDKXSW 对应的值班人是 值班人-7581，编号 REF-HH25AV 对应的值班人是 值班人-9509。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-QEW6HW 对应的值班人是 值班人-3683，编号 REF-VPML2S 对应的值班人是 值班人-5200。",
"written": 5,
"correct": true,
"matched": [
"值班人-5200"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-8L9J6U 对应的值班人是 值班人 -8439，编号 REF-CMJ4GB 对应的值班人是 值班人 -5479。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-79PKV7 对应的值班人是 值班人-2930，编号 REF-DY6T58 对应的值班人是 值班人-3131。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-4BVM4J 对应的值班人是 值班人 -1683，编号 REF-MM463F 对应的值班人是 值班人 -4978。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-AZTA7R 对应的值班人是 值班人 -5244。编号 REF-QA25ZU 对应的值班人是 值班人 -6732。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-PJUBBR 对应的值班人是 值班人-9282，编号 REF-R3TAN6 对应的值班人是 值班人-8974。",
"written": 5,
"correct": true,
"matched": [
"值班人-9282"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-KEZE8S 对应的值班人是 值班人 -2662，编号 REF-A4AKAV 对应的值班人是 值班人 -2212。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-LQB5P8 对应的值班人是 值班人 -1426，编号 REF-QQGRV5 对应的值班人是 值班人 -3660。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "multi_hop",
"query": "负责那个项目的同事对应的值班人是谁？",
"reply": "编号 REF-PBKXTM 对应的值班人是 值班人-5202，编号 REF-8X8Q8C 对应的值班人是 值班人-7783。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我随身那台设备的末几位数字是啥？",
"reply": "您的手机尾号是分机 6612。",
"written": 4,
"correct": true,
"matched": [
"分机 6612"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我随身那台设备的末几位数字是啥？",
"reply": "您的手机尾号是 10.20.3.7:5432，您的座机尾号是 每周一 09:30。",
"written": 4,
"correct": true,
"matched": [
"10.20.3.7:5432"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我白天上班待的地儿在哪儿？",
"reply": "您的办公城市是 v1.9.7。",
"written": 4,
"correct": true,
"matched": [
"v1.9.7"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我白天上班待的地儿在哪儿？",
"reply": "您的办公城市是 256GB，您的常住城市是 A-7719。",
"written": 4,
"correct": true,
"matched": [
"256GB"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我白天上班待的地儿在哪儿？",
"reply": "您的办公城市是 D-9021，常住城市是分机 6612。",
"written": 4,
"correct": true,
"matched": [
"D-9021"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我随身那台设备的末几位数字是啥？",
"reply": "您的随身设备末几位数字是：user；座机尾号；李工 139****3344。",
"written": 4,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我随身那台设备的末几位数字是啥？",
"reply": "您的手机尾号是 91.5%，座机尾号是 78%。",
"written": 4,
"correct": true,
"matched": [
"91.5%"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我白天上班待的地儿在哪儿？",
"reply": "您的办公城市是 78%。",
"written": 4,
"correct": true,
"matched": [
"78%"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我随身那台设备的末几位数字是啥？",
"reply": "您的座机尾号是赵工 137****8890，手机尾号是 v2.8.0。",
"written": 4,
"correct": true,
"matched": [
"v2.8.0"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我白天上班待的地儿在哪儿？",
"reply": "您的办公城市是 钱工 135****2211。",
"written": 4,
"correct": true,
"matched": [
"钱工 135****2211"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我白天上班待的地儿在哪儿？",
"reply": "关于您白天上班待的地儿，当前长期记忆中未包含该信息。已知信息如下：user；主库地址；v1.9.7。",
"written": 4,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我随身那台设备的末几位数字是啥？",
"reply": "您的手机尾号是 v4.0.0-rc1。",
"written": 4,
"correct": true,
"matched": [
"v4.0.0-rc1"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我白天上班待的地儿在哪儿？",
"reply": "您的办公城市是 E-4408。",
"written": 4,
"correct": true,
"matched": [
"E-4408"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我随身那台设备的末几位数字是啥？",
"reply": "我目前无法回答您关于随身设备末几位数字的问题，因为长期记忆中未包含该设备的任何信息。",
"written": 4,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我白天上班待的地儿在哪儿？",
"reply": "关于您白天上班待的地儿，当前长期记忆中未包含相关信息，证据不足，不知道。",
"written": 4,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": true
},
{
"category": "near_miss",
"query": "我随身那台设备的末几位数字是啥？",
"reply": "您的手机尾号是 李工 139****3344，您的座机尾号是 32GB。",
"written": 4,
"correct": true,
"matched": [
"李工 139****3344"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我随身那台设备的末几位数字是啥？",
"reply": "您的座机尾号是 v2.8.0，手机尾号是 v4.0.0-rc1。",
"written": 4,
"correct": true,
"matched": [
"v4.0.0-rc1"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我随身那台设备的末几位数字是啥？",
"reply": "您的手机尾号是 91.5%，您的座机尾号是 B-3310。",
"written": 4,
"correct": true,
"matched": [
"91.5%"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我随身那台设备的末几位数字是啥？",
"reply": "您的手机尾号是 10.20.9.1:6379。",
"written": 4,
"correct": true,
"matched": [
"10.20.9.1:6379"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我白天上班待的地儿在哪儿？",
"reply": "关于您白天上班待的地儿，当前长期记忆中未包含相关信息，无法回答。",
"written": 4,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我随身那台设备的末几位数字是啥？",
"reply": "您的手机尾号是 v1.9.7，座机尾号是 D-9021。",
"written": 4,
"correct": true,
"matched": [
"v1.9.7"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我随身那台设备的末几位数字是啥？",
"reply": "您的手机尾号是 63.2%，座机尾号是 B3-204。",
"written": 4,
"correct": true,
"matched": [
"63.2%"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我白天上班待的地儿在哪儿？",
"reply": "您的办公城市是 32GB。您的常住城市是 v4.0.0-rc1。",
"written": 4,
"correct": true,
"matched": [
"32GB"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我随身那台设备的末几位数字是啥？",
"reply": "您的手机尾号是 256GB。",
"written": 4,
"correct": true,
"matched": [
"256GB"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "near_miss",
"query": "我随身那台设备的末几位数字是啥？",
"reply": "您的设备末几位数字是 v2.8.0。",
"written": 4,
"correct": true,
"matched": [
"v2.8.0"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我东西放在哪一格？",
"reply": "您的仓库库位是 85%。",
"written": 9,
"correct": true,
"matched": [
"85%"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我提交到哪个地方？",
"reply": "您提交到哪个地方取决于具体指代的内容（如文件、数据、任务等），当前对话中未提供明确上下文，无法确定具体位置。若涉及编号或路径，请补充说明以便准确回答。",
"written": 9,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我保障什么时候结束？",
"reply": "您的保险到期是 赵工 137****8890。",
"written": 9,
"correct": true,
"matched": [
"赵工 137****8890"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我货是谁供的？",
"reply": "您的供应商是 user，供应商，63.2%。",
"written": 9,
"correct": true,
"matched": [
"63.2%"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我纸面上那个号是多少？",
"reply": "您的合同编号是 B-3310。",
"written": 9,
"correct": true,
"matched": [
"B-3310"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我这周按什么表轮？",
"reply": "您的值班表是每周三 22:00。",
"written": 9,
"correct": true,
"matched": [
"每周三 22:00"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我到多少会触发报警？",
"reply": "您未提供具体的报警阈值参数或设备型号，无法确定触发报警的具体数值。请补充说明设备类型、监测指标（如温度、压力等）或相关配置信息，以便我提供准确答案。",
"written": 9,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我协议编号是什么？",
"reply": "您的协议编号是 v4.0.0-rc1。",
"written": 9,
"correct": true,
"matched": [
"v4.0.0-rc1"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我这周按什么表轮？",
"reply": "您的值班表是 10.20.3.7:5432。",
"written": 9,
"correct": true,
"matched": [
"10.20.3.7:5432"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我货码在哪个位置？",
"reply": "您的货码在 32GB 仓库库位。",
"written": 9,
"correct": true,
"matched": [
"32GB"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我 weekly 的会安排在何时？",
"reply": "您的周会时间是 v2.8.0。",
"written": 9,
"correct": true,
"matched": [
"v2.8.0"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我数据落在哪一个地址？",
"reply": "您的主库地址是 D-9021。",
"written": 9,
"correct": true,
"matched": [
"D-9021"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我一般什么时候做上线？",
"reply": "您的发布窗口是 王工 138****6621。",
"written": 9,
"correct": true,
"matched": [
"王工 138****6621"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我这周按什么表轮？",
"reply": "您的值班表是 B-3310。",
"written": 9,
"correct": true,
"matched": [
"B-3310"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我协议编号是什么？",
"reply": "您的协议编号是 李工 139****3344。",
"written": 9,
"correct": true,
"matched": [
"李工 139****3344"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我上手需要多长时间？",
"reply": "关于您上手所需的时间，当前长期记忆中未包含此信息，无法回答。",
"written": 9,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我哪条线一破就叫？",
"reply": "您提到的“哪条线一破就叫？”缺乏具体上下文，无法确定您指的是哪条线路或事件，因此不知道具体答案。如果您能提供更多细节，例如涉及的具体线路名称或事件背景，我将尽力为您提供准确信息",
"written": 9,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": true
},
{
"category": "noise_context",
"query": "我一般什么时候做上线？",
"reply": "关于您一般什么时候做上线，当前长期记忆中无相关记录，无法回答。",
"written": 9,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我这周按什么表轮？",
"reply": "您的值班表是 v3.14.2。",
"written": 9,
"correct": true,
"matched": [
"v3.14.2"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我团队什么时候碰一次？",
"reply": "您的团队周会时间是 D-9021。",
"written": 9,
"correct": true,
"matched": [
"D-9021"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我东西放在哪一格？",
"reply": "您的仓库库位是 10.30.1.9:5432。",
"written": 9,
"correct": true,
"matched": [
"10.30.1.9:5432"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我提交到哪个地方？",
"reply": "您提交到 N-QG46HPKB。",
"written": 9,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我提交到哪个地方？",
"reply": "您未提供具体提交内容或上下文，无法确定提交地点。请补充说明提交对象（如文件、数据、任务等）及场景，以便准确回答。",
"written": 9,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我纸面上那个号是多少？",
"reply": "您的合同编号是 夜间 02:00。",
"written": 9,
"correct": true,
"matched": [
"夜间 02:00"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "noise_context",
"query": "我货码在哪个位置？",
"reply": "您的货码位置是：user；仓库库位；赵工 137****8890。",
"written": 9,
"correct": true,
"matched": [
"赵工 137****8890"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我这份多久后失效？",
"reply": "您的培训周期是 10.30.1.9:5432，您的合同编号是 钱工 135****2211。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我写库连的是哪里？",
"reply": "您的代码仓库是 91.5%，仓库库位是 E-4408。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我 weekly 的会安排在何时？",
"reply": "关于您 weekly 的会安排时间，当前长期记忆中未包含相关信息，无法回答。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我 weekly 的会安排在何时？",
"reply": "关于您 weekly 的会安排时间，当前长期记忆中未包含此信息，无法回答。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我到多少会触发报警？",
"reply": "关于触发报警的阈值，当前长期记忆中未包含相关信息。已知事实包括：user 的值班电话是 v1.9.7，培训周期是 B3-204。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我轮班时该拨什么号码？",
"reply": "关于轮班时该拨的号码，当前长期记忆中未包含此信息，无法提供具体号码。已知信息仅涉及：user；合同编号；赵工 137****8890；user；培训周期",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我这周按什么表轮？",
"reply": "关于您这周按什么表轮，当前长期记忆中未包含该信息，无法回答。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我保障什么时候结束？",
"reply": "您的培训周期是 32GB。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我排班依据是哪一份？",
"reply": "我排班依据是：user；周会时间；E-4408；仓库库位；每周三 22:00。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我协议编号是什么？",
"reply": "您的协议编号未知。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我这周按什么表轮？",
"reply": "关于您这周按什么表轮，当前长期记忆中未包含该信息。已知事实仅包括：user；主库地址；每周三 22:00；user；合同编号；凌晨 03:3",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我这周按什么表轮？",
"reply": "关于您这周按什么表轮，当前长期记忆中未包含该信息。已知信息如下：user；代码仓库；10.20.3.7:5432。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我 weekly 的会安排在何时？",
"reply": "您的培训周期是每周三 22:00。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我源码托管在哪里？",
"reply": "关于源码托管位置，当前长期记忆中未包含相关信息，无法回答。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我一般什么时候做上线？",
"reply": "您的值班表是 夜间 02:00。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我保障什么时候结束？",
"reply": "您的值班表是 夜间 02:00。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我写库连的是哪里？",
"reply": "您的代码仓库是 每周三 22:00。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我上手需要多长时间？",
"reply": "关于上手所需时间，当前长期记忆中未包含相关信息，无法回答。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我这份多久后失效？",
"reply": "您的培训周期是 分机 6612，周会时间是 夜间 02:00。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我数据落在哪一个地址？",
"reply": "关于您的数据地址，当前长期记忆中未包含相关信息，无法回答。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我东西放在哪一格？",
"reply": "关于您物品存放的具体位置，当前长期记忆中未包含该信息，无法回答。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我排班依据是哪一份？",
"reply": "关于排班依据的具体文件，当前长期记忆中未包含相关信息，无法回答。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我到多少会触发报警？",
"reply": "关于报警触发条件，当前记忆中没有相关记录。已知信息：user；保险到期；A-7719。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我轮班时该拨什么号码？",
"reply": "您的值班表号码是 B3-204。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "unknown_attribute",
"query": "我提交到哪个地方？",
"reply": "您的仓库库位是 每周一 09:30。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我提交到哪个地方？",
"reply": "您的代码提交到 D-9021。user；代码仓库；D-9021",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我上游是哪一家？",
"reply": "您的上游供应商是 B-3310。\nuser；供应商；B-3310",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我这份多久后失效？",
"reply": "您的保险到期是每周一 09:30。user；保险到期；每周一 09:30",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我货是谁供的？",
"reply": "您的货由王工 138****6621 供应。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我提交到哪个地方？",
"reply": "我提交到 D-9021。user；代码仓库；D-9021",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我上手需要多长时间？",
"reply": "我训 256GB 那么久。",
"written": 5,
"correct": true,
"matched": [
"256GB"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我哪条线一破就叫？",
"reply": "您的告警阈值是每周三 22:00。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我数据落在哪一个地址？",
"reply": "我主库落在 钱工 135****2211。",
"written": 5,
"correct": true,
"matched": [
"钱工 135****2211"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我一般什么时候做上线？",
"reply": "我一般固定 128GB 做上线。",
"written": 5,
"correct": true,
"matched": [
"128GB"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我纸面上那个号是多少？",
"reply": "我签的是 64GB 号合同。",
"written": 5,
"correct": true,
"matched": [
"64GB"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我一般什么时候做上线？",
"reply": "我固定每周三 22:00 做上线。",
"written": 5,
"correct": true,
"matched": [
"每周三 22:00"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我一般什么时候做上线？",
"reply": "我一般固定 10.30.1.9:5432 做上线。",
"written": 5,
"correct": true,
"matched": [
"10.30.1.9:5432"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我哪条线一破就叫？",
"reply": "您的告警阈值是 钱工 135****2211。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我挑哪个时段变更？",
"reply": "关于变更时段，当前长期记忆中未包含具体信息，证据不足，明确说不知道。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": true
},
{
"category": "update_conflict",
"query": "我东西放在哪一格？",
"reply": "您的东西放在每周三 22:00。实体：user；仓库库位：每周三 22:00。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我到多少会触发报警？",
"reply": "我超过 B-3310 就报警。",
"written": 5,
"correct": true,
"matched": [
"B-3310"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我哪条线一破就叫？",
"reply": "您的告警阈值是每周三 22:00。user；告警阈值；每周三 22:00",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我东西放在哪一格？",
"reply": "您的东西放在 63.2% 的仓库库位。user 的仓库库位是 63.2%。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我这份多久后失效？",
"reply": "您的保险到期时间是每周一 09:30。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我要学多久？",
"reply": "我训 v1.9.7 那么久。",
"written": 5,
"correct": true,
"matched": [
"v1.9.7"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我到多少会触发报警？",
"reply": "我超过 64GB 就报警。",
"written": 5,
"correct": true,
"matched": [
"64GB"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我货码在哪个位置？",
"reply": "您的货码位于仓库库位：夜间 02:00。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我货是谁供的？",
"reply": "您的供应商是 128GB。",
"written": 5,
"correct": false,
"matched": [],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我要学多久？",
"reply": "我训每周三 22:00 那么久。",
"written": 5,
"correct": true,
"matched": [
"每周三 22:00"
],
"abstained": false,
"wrongly_abstained": false
},
{
"category": "update_conflict",
"query": "我源码托管在哪里？",
"reply": "我源码放在 王工 138****6621。",
"written": 5,
"correct": true,
"matched": [
"王工 138****6621"
],
"abstained": false,
"wrongly_abstained": false
}
]
}
}
Reference in New Issue View Git Blame Copy Permalink