Natural Memory NM2.1: 记忆路由器分叉、数据集缺陷修复与全轴评测证据
- 引入 MemoryRouterXL 与 v5/v6 流式多线程训练/编码管线 - 修复 prepare_memory_router_dataset 候选池重建缺陷(mega 家族 3568x 加速,输出逐字节相同) - 修复 v5 被破坏的拒答与多跳标签(train 未知样本 319 -> 16319,multi_hop 平均正例 1.00 -> 2.00) - 同存储预算下 V2-128 v6 逐轴 22/22 通过:Top-1 41.12% -> 94.62%,未知拒答 0.00% -> 100.00% - 记录三条被实测推翻的显然优化(logits_to_keep=1 反而慢 55%、XL 容量未带来收益) - 记忆手术跨架构可移植性 14/14,读写关闭时与原生模型逐位相同
This commit is contained in:
@@ -0,0 +1,728 @@
|
||||
{
|
||||
"eval_file": "data/zero_overlap/eval.jsonl",
|
||||
"episodes": 50,
|
||||
"candidate_count": 24,
|
||||
"router": "checkpoints\\router_replay_v7_v2_128\\memory_router_v2.pt",
|
||||
"results": {
|
||||
"0.00": {
|
||||
"router": "blend=0.00",
|
||||
"cases": 50,
|
||||
"answerable_cases": 0,
|
||||
"unknown_cases": 50,
|
||||
"accuracy_pct": 0.0,
|
||||
"wrong_attribute_pct": 0.0,
|
||||
"unknown_leak_pct": 78.0,
|
||||
"mean_records_selected": 6.88,
|
||||
"records_active_min": 23,
|
||||
"records_active_max": 23,
|
||||
"stop_reasons": {
|
||||
"evidence_found": 50
|
||||
},
|
||||
"seconds": 320.3
|
||||
}
|
||||
},
|
||||
"rows": {
|
||||
"0.00": [
|
||||
{
|
||||
"id": "zero-overlap-000106",
|
||||
"attribute": "咖啡口味",
|
||||
"answerable": false,
|
||||
"expected": "VAL-837DYYWV",
|
||||
"reply": "关于您的闹钟设置时间,当前长期记忆中未包含相关证据,无法回答。已知信息包括:user;闹钟设置时间;未知。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": false,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000045",
|
||||
"attribute": "办公城市",
|
||||
"answerable": false,
|
||||
"expected": "VAL-CG4X6ZWD",
|
||||
"reply": "您的办公城市是 VAL-SZ4R5GT3。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000246",
|
||||
"attribute": "常住城市",
|
||||
"answerable": false,
|
||||
"expected": "VAL-6BWJYDE4",
|
||||
"reply": "关于您小时候待的地方,当前长期记忆中未包含此信息。已知信息如下:user;常住城市;VAL-6BWJYDE4;办公城市;VAL-88V99MPM;宿舍楼号",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000244",
|
||||
"attribute": "午餐偏好",
|
||||
"answerable": false,
|
||||
"expected": "VAL-2VLHC26V",
|
||||
"reply": "关于您服务的对象,当前长期记忆中未包含相关信息,无法回答。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": false,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000180",
|
||||
"attribute": "客户名称",
|
||||
"answerable": false,
|
||||
"expected": "VAL-3F87H6UT",
|
||||
"reply": "您随身那台设备的末端数字是 VAL-X5VZ48QF。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 1
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000144",
|
||||
"attribute": "入职年份",
|
||||
"answerable": false,
|
||||
"expected": "VAL-GLHQMRN6",
|
||||
"reply": "关于您睡觉的地儿是第几排,当前长期记忆中未包含此信息。已知信息如下:user;档案标识;VAL-AVX95V7L;手机尾号;VAL-LBRPQJRT",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000051",
|
||||
"attribute": "项目代号",
|
||||
"answerable": false,
|
||||
"expected": "VAL-KG4RZYH5",
|
||||
"reply": "您的办公城市是 VAL-W9KGRD9U。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000173",
|
||||
"attribute": "客户名称",
|
||||
"answerable": false,
|
||||
"expected": "VAL-P84HTKZR",
|
||||
"reply": "您随身那台设备的末端数字是 Y。\n设备型号:VAL-NARXVSSY",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 1
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000261",
|
||||
"attribute": "课程名称",
|
||||
"answerable": false,
|
||||
"expected": "VAL-Q6ZMN3XU",
|
||||
"reply": "关于您的上级信息,当前长期记忆中未包含相关记录。已知信息包括:紧急联系人姓氏是 VAL-QT3YM55W;客户名称是 VAL-7VDS2BRU;默认语言是 VAL-QB",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000195",
|
||||
"attribute": "午餐偏好",
|
||||
"answerable": false,
|
||||
"expected": "VAL-LHKPHTRT",
|
||||
"reply": "关于您坐在哪一间房的信息,当前长期记忆中未包含该数据。已知信息包括:user;宿舍楼号;VAL-JCMMXZA3。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000082",
|
||||
"attribute": "项目代号",
|
||||
"answerable": false,
|
||||
"expected": "VAL-7Z642QXK",
|
||||
"reply": "您的白天上班地点(工位楼层)是 VAL-4ZSWT4QA。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000177",
|
||||
"attribute": "咖啡口味",
|
||||
"answerable": false,
|
||||
"expected": "VAL-D8QSWC9M",
|
||||
"reply": "关于您白天的午餐打算,当前长期记忆中未包含此信息。已知信息如下:user;课程名称;VAL-YJK3DDHQ;通勤方式;VAL-YYRR5WV6;入职年份;VAL-G",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000191",
|
||||
"attribute": "通勤方式",
|
||||
"answerable": false,
|
||||
"expected": "VAL-NZGT22WC",
|
||||
"reply": "您的白天上班地点(工位楼层)是 VAL-QAYPWLSW。user;工位楼层;VAL-QAYPWLSW",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000003",
|
||||
"attribute": "办公城市",
|
||||
"answerable": false,
|
||||
"expected": "VAL-Q7W2CKC6",
|
||||
"reply": "关于您使用的机器档位,当前长期记忆中未包含此信息,证据不足,不知道。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": false,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 1
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000297",
|
||||
"attribute": "档案标识",
|
||||
"answerable": false,
|
||||
"expected": "VAL-CKF3THHA",
|
||||
"reply": "您的团队名称是 VAL-PZ8Q4Y45。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000042",
|
||||
"attribute": "常住城市",
|
||||
"answerable": false,
|
||||
"expected": "VAL-VA2XR4ND",
|
||||
"reply": "您的默认语言是 VAL-W4NLC6Y6,您的出生城市是 VAL-6AVTPXZE,您的课程名称是 VAL-59JAQXQE,您的起床时间是 VAL-W56V6ELE",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000022",
|
||||
"attribute": "运动习惯",
|
||||
"answerable": false,
|
||||
"expected": "VAL-7RG27UUU",
|
||||
"reply": "关于您服务的对象,当前长期记忆中未包含相关信息,无法回答。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": false,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000077",
|
||||
"attribute": "咖啡口味",
|
||||
"answerable": false,
|
||||
"expected": "VAL-MH645VGQ",
|
||||
"reply": "您随身那台设备的末端数字是 VAL-N9RQGGEM。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 1
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000186",
|
||||
"attribute": "阅读工具",
|
||||
"answerable": false,
|
||||
"expected": "VAL-LSSKXTKX",
|
||||
"reply": "关于您最近报名的课程内容,当前长期记忆中未包含相关证据,无法回答。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": false,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000095",
|
||||
"attribute": "团队名称",
|
||||
"answerable": false,
|
||||
"expected": "VAL-9XSGFFKT",
|
||||
"reply": "您的默认语言是 VAL-LBW9VEU5,课程名称是 VAL-DEU57W65,出生城市是 VAL-3T7B5W22,起床时间是 VAL-9BGXWX2",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000200",
|
||||
"attribute": "手机尾号",
|
||||
"answerable": false,
|
||||
"expected": "VAL-87U9573K",
|
||||
"reply": "关于您使用的机器档位,当前长期记忆中未包含此信息,证据不足,无法回答。已知事实仅涉及您的常用编辑器:user;常用编辑器;VAL-T8QM27AS。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 1
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000134",
|
||||
"attribute": "运动习惯",
|
||||
"answerable": false,
|
||||
"expected": "VAL-K2XP56JM",
|
||||
"reply": "您的团队名称是 VAL-TYFQ7RE2。user;团队名称;VAL-TYFQ7RE2",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000156",
|
||||
"attribute": "手机尾号",
|
||||
"answerable": false,
|
||||
"expected": "VAL-LT5AGPC2",
|
||||
"reply": "您的紧急联系人姓氏是 VAL-LGUH8DRR,团队名称是 VAL-5PR4ZXGX,客户名称是 VAL-JHNXGADZ,阅读工具是 VAL-UQXK9AZ7,课程",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000014",
|
||||
"attribute": "课程名称",
|
||||
"answerable": false,
|
||||
"expected": "VAL-BYHPZURM",
|
||||
"reply": "您的团队名称是 user;团队名称;VAL-CC9Z7XVY。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000049",
|
||||
"attribute": "午餐偏好",
|
||||
"answerable": false,
|
||||
"expected": "VAL-2SW8J6P7",
|
||||
"reply": "关于您所在的小组信息,当前长期记忆中未包含相关记录。已知信息包括:user;课程名称;VAL-RCSW5FHA;user;紧急联系人姓氏;VAL-JTC9REXD;user;宿舍",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000233",
|
||||
"attribute": "默认语言",
|
||||
"answerable": false,
|
||||
"expected": "VAL-7Z7GKSVP",
|
||||
"reply": "您的团队名称是 VAL-ZEQ9YFZ7。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000133",
|
||||
"attribute": "紧急联系人姓氏",
|
||||
"answerable": false,
|
||||
"expected": "VAL-7L7Q38BR",
|
||||
"reply": "您的白天上班地点(工位楼层)是 VAL-UY99NYLA。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000289",
|
||||
"attribute": "通勤方式",
|
||||
"answerable": false,
|
||||
"expected": "VAL-MPCP37YU",
|
||||
"reply": "关于您翻书时依赖的具体习惯或偏好,当前长期记忆中未包含相关记录。已知信息包括:user;课程名称;VAL-BB9VQBPD;user;运动习惯;VAL-VDC24N9",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000251",
|
||||
"attribute": "入职年份",
|
||||
"answerable": false,
|
||||
"expected": "VAL-MZAMG4PA",
|
||||
"reply": "关于您翻书时依赖的具体习惯或偏好,当前长期记忆中未包含相关记录。已知信息包括:user;课程名称;VAL-6XNPRKHU;运动习惯;VAL-SU7SPKUL;",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000272",
|
||||
"attribute": "手机尾号",
|
||||
"answerable": false,
|
||||
"expected": "VAL-G6R4SSAR",
|
||||
"reply": "我的项目代号是 VAL-YKGHW6B2。我的阅读工具是 VAL-28ZAPDW7。我的课程名称是 VAL-GKMBTT38。我的团队名称是 VAL-5SH59B",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000162",
|
||||
"attribute": "运动习惯",
|
||||
"answerable": false,
|
||||
"expected": "VAL-XX4BJ68R",
|
||||
"reply": "根据长期记忆证据,您的关键联系信息如下:\n- 默认语言:VAL-T6Z8A8Y9\n- 课程名称:VAL-RCG6KT7B\n- 出生城市:VAL-V",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000222",
|
||||
"attribute": "档案标识",
|
||||
"answerable": false,
|
||||
"expected": "VAL-QC9CSDPT",
|
||||
"reply": "您的交流语言未在当前长期记忆中记录,无法回答。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": false,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000020",
|
||||
"attribute": "运动习惯",
|
||||
"answerable": false,
|
||||
"expected": "VAL-4WR8CJPQ",
|
||||
"reply": "关于您坐在哪一间房的信息,当前长期记忆中未包含此数据,无法回答。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": false,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000039",
|
||||
"attribute": "团队名称",
|
||||
"answerable": false,
|
||||
"expected": "VAL-3PQ7DMM4",
|
||||
"reply": "关于您翻书时依赖的具体习惯或偏好,当前长期记忆中未包含相关记录。已知信息包括:user;课程名称;VAL-WD9K3FT3;user;运动习惯;VAL-JUZ426",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000066",
|
||||
"attribute": "宿舍楼号",
|
||||
"answerable": false,
|
||||
"expected": "VAL-F9CBCKRA",
|
||||
"reply": "收信时您的地址后半段是 VAL-ACS4FE7W。\n其他已保存信息如下:\n- 出生城市:VAL-ACS4FE7W\n- 宿舍楼号:VAL-F9CBCK",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000292",
|
||||
"attribute": "办公城市",
|
||||
"answerable": false,
|
||||
"expected": "VAL-Z57S5YFH",
|
||||
"reply": "收信时您的地址后半段是 VAL-BSR6K6QU。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000094",
|
||||
"attribute": "设备型号",
|
||||
"answerable": false,
|
||||
"expected": "VAL-WNVEZKXA",
|
||||
"reply": "您的默认语言是 VAL-GELDDSM7,课程名称是 VAL-859DKJCX,出生城市是 VAL-YXYKCVJ8,起床时间是 VAL-CAVECGSD,午餐偏好是 VAL-",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000166",
|
||||
"attribute": "运动习惯",
|
||||
"answerable": false,
|
||||
"expected": "VAL-J453FW7H",
|
||||
"reply": "关于您何时开始在这家公司上班,当前记忆中没有相关记录,无法提供确切答案。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": false,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 1
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000249",
|
||||
"attribute": "团队名称",
|
||||
"answerable": false,
|
||||
"expected": "VAL-8MJTBPJX",
|
||||
"reply": "收信时地址的后半段是 VAL-VM6X8APT。user;常住城市;VAL-VM6X8APT",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000131",
|
||||
"attribute": "工位楼层",
|
||||
"answerable": false,
|
||||
"expected": "VAL-WA8VB5MZ",
|
||||
"reply": "根据长期记忆证据,您的默认语言是 VAL-5E9TSP8M,出生城市是 VAL-B6YYHYN5,起床时间是 VAL-E4APPRTR,课程名称是 VAL-9FLEK",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000029",
|
||||
"attribute": "项目代号",
|
||||
"answerable": false,
|
||||
"expected": "VAL-JKN8YNS8",
|
||||
"reply": "我的工位楼层是 VAL-DL6VUFV8。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000286",
|
||||
"attribute": "入职年份",
|
||||
"answerable": false,
|
||||
"expected": "VAL-HZ3H82PN",
|
||||
"reply": "关于您的闹钟设置时间,当前长期记忆中未包含该信息。已知信息如下:user;设备型号;VAL-2VHYRYJP;入职年份;VAL-HCW4XTZP;常住城市;VAL-XUV",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000264",
|
||||
"attribute": "紧急联系人姓氏",
|
||||
"answerable": false,
|
||||
"expected": "VAL-HJX2VWU7",
|
||||
"reply": "关于您每天如何到公司,当前长期记忆中未包含交通方式或通勤路线的相关证据。\nuser;办公城市;VAL-XP2M992C\nuser;常住城市;VAL-WRER4CPX",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000224",
|
||||
"attribute": "阅读工具",
|
||||
"answerable": false,
|
||||
"expected": "VAL-CP9SFPRU",
|
||||
"reply": "关于您使用的机器档位,当前长期记忆中未包含此信息,证据不足,无法回答。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": false,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 1
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000254",
|
||||
"attribute": "紧急联系人姓氏",
|
||||
"answerable": false,
|
||||
"expected": "VAL-YA4DYD4T",
|
||||
"reply": "关于您何时开始在这家公司上班,当前记忆中没有相关记录。已知信息:user;办公城市;VAL-T7WB8GDW。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 1
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000291",
|
||||
"attribute": "常住城市",
|
||||
"answerable": false,
|
||||
"expected": "VAL-36J895QM",
|
||||
"reply": "关于您的上级信息,当前长期记忆中未包含相关记录。已知信息包括:user;紧急联系人姓氏;VAL-9SAUTM24;客户名称;VAL-AMK2PGQX;默认语言;VAL",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000287",
|
||||
"attribute": "课程名称",
|
||||
"answerable": false,
|
||||
"expected": "VAL-9FVF5F7K",
|
||||
"reply": "关于您翻书时依赖的具体习惯或偏好,当前长期记忆中未包含相关记录。已知信息包括:user;课程名称;VAL-AGGV7KJD;运动习惯;VAL-NAAS25FZ;",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000167",
|
||||
"attribute": "常住城市",
|
||||
"answerable": false,
|
||||
"expected": "VAL-E5HUQXKW",
|
||||
"reply": "关于您睡觉的地儿是第几排,当前长期记忆中未包含此信息,无法回答。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": false,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000058",
|
||||
"attribute": "咖啡口味",
|
||||
"answerable": false,
|
||||
"expected": "VAL-A86MFKEH",
|
||||
"reply": "关于您翻书时依赖的具体习惯或偏好,当前长期记忆中未包含相关信息。已知信息如下:user;翻书依赖;未知。",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": false,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
},
|
||||
{
|
||||
"id": "zero-overlap-000151",
|
||||
"attribute": "运动习惯",
|
||||
"answerable": false,
|
||||
"expected": "VAL-PHKP82V7",
|
||||
"reply": "关于您小时候待的地方,当前长期记忆中未包含此信息。已知信息如下:user;常住城市;VAL-Q9M7JUXB;办公城市;VAL-MRKQZ6JF;宿舍楼号;",
|
||||
"correct": false,
|
||||
"wrong_attribute": false,
|
||||
"leaked": true,
|
||||
"records_written": 24,
|
||||
"records_active": 23,
|
||||
"stop_reason": "evidence_found",
|
||||
"records_selected": 8
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user