Natural Memory NM2.1: 记忆路由器分叉、数据集缺陷修复与全轴评测证据

- 引入 MemoryRouterXL 与 v5/v6 流式多线程训练/编码管线
- 修复 prepare_memory_router_dataset 候选池重建缺陷(mega 家族 3568x 加速,输出逐字节相同)
- 修复 v5 被破坏的拒答与多跳标签(train 未知样本 319 -> 16319,multi_hop 平均正例 1.00 -> 2.00)
- 同存储预算下 V2-128 v6 逐轴 22/22 通过:Top-1 41.12% -> 94.62%,未知拒答 0.00% -> 100.00%
- 记录三条被实测推翻的显然优化(logits_to_keep=1 反而慢 55%、XL 容量未带来收益)
- 记忆手术跨架构可移植性 14/14,读写关闭时与原生模型逐位相同
This commit is contained in:
WpyQwq
2026-09-19 11:11:31 +08:00
commit 643e22ecb9
484 changed files with 306821 additions and 0 deletions
+728
View File
@@ -0,0 +1,728 @@
{
"eval_file": "data/zero_overlap/eval.jsonl",
"episodes": 50,
"candidate_count": 24,
"router": "checkpoints\\router_replay_v7_v2_128\\memory_router_v2.pt",
"results": {
"0.00": {
"router": "blend=0.00",
"cases": 50,
"answerable_cases": 0,
"unknown_cases": 50,
"accuracy_pct": 0.0,
"wrong_attribute_pct": 0.0,
"unknown_leak_pct": 78.0,
"mean_records_selected": 6.88,
"records_active_min": 23,
"records_active_max": 23,
"stop_reasons": {
"evidence_found": 50
},
"seconds": 320.3
}
},
"rows": {
"0.00": [
{
"id": "zero-overlap-000106",
"attribute": "咖啡口味",
"answerable": false,
"expected": "VAL-837DYYWV",
"reply": "关于您的闹钟设置时间,当前长期记忆中未包含相关证据,无法回答。已知信息包括:user;闹钟设置时间;未知。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000045",
"attribute": "办公城市",
"answerable": false,
"expected": "VAL-CG4X6ZWD",
"reply": "您的办公城市是 VAL-SZ4R5GT3。",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000246",
"attribute": "常住城市",
"answerable": false,
"expected": "VAL-6BWJYDE4",
"reply": "关于您小时候待的地方,当前长期记忆中未包含此信息。已知信息如下:user;常住城市;VAL-6BWJYDE4;办公城市;VAL-88V99MPM;宿舍楼号",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000244",
"attribute": "午餐偏好",
"answerable": false,
"expected": "VAL-2VLHC26V",
"reply": "关于您服务的对象,当前长期记忆中未包含相关信息,无法回答。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000180",
"attribute": "客户名称",
"answerable": false,
"expected": "VAL-3F87H6UT",
"reply": "您随身那台设备的末端数字是 VAL-X5VZ48QF。",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 1
},
{
"id": "zero-overlap-000144",
"attribute": "入职年份",
"answerable": false,
"expected": "VAL-GLHQMRN6",
"reply": "关于您睡觉的地儿是第几排,当前长期记忆中未包含此信息。已知信息如下:user;档案标识;VAL-AVX95V7L;手机尾号;VAL-LBRPQJRT",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000051",
"attribute": "项目代号",
"answerable": false,
"expected": "VAL-KG4RZYH5",
"reply": "您的办公城市是 VAL-W9KGRD9U。",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000173",
"attribute": "客户名称",
"answerable": false,
"expected": "VAL-P84HTKZR",
"reply": "您随身那台设备的末端数字是 Y。\n设备型号:VAL-NARXVSSY",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 1
},
{
"id": "zero-overlap-000261",
"attribute": "课程名称",
"answerable": false,
"expected": "VAL-Q6ZMN3XU",
"reply": "关于您的上级信息,当前长期记忆中未包含相关记录。已知信息包括:紧急联系人姓氏是 VAL-QT3YM55W;客户名称是 VAL-7VDS2BRU;默认语言是 VAL-QB",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000195",
"attribute": "午餐偏好",
"answerable": false,
"expected": "VAL-LHKPHTRT",
"reply": "关于您坐在哪一间房的信息,当前长期记忆中未包含该数据。已知信息包括:user;宿舍楼号;VAL-JCMMXZA3。",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000082",
"attribute": "项目代号",
"answerable": false,
"expected": "VAL-7Z642QXK",
"reply": "您的白天上班地点(工位楼层)是 VAL-4ZSWT4QA。",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000177",
"attribute": "咖啡口味",
"answerable": false,
"expected": "VAL-D8QSWC9M",
"reply": "关于您白天的午餐打算,当前长期记忆中未包含此信息。已知信息如下:user;课程名称;VAL-YJK3DDHQ;通勤方式;VAL-YYRR5WV6;入职年份;VAL-G",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000191",
"attribute": "通勤方式",
"answerable": false,
"expected": "VAL-NZGT22WC",
"reply": "您的白天上班地点(工位楼层)是 VAL-QAYPWLSW。user;工位楼层;VAL-QAYPWLSW",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000003",
"attribute": "办公城市",
"answerable": false,
"expected": "VAL-Q7W2CKC6",
"reply": "关于您使用的机器档位,当前长期记忆中未包含此信息,证据不足,不知道。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 1
},
{
"id": "zero-overlap-000297",
"attribute": "档案标识",
"answerable": false,
"expected": "VAL-CKF3THHA",
"reply": "您的团队名称是 VAL-PZ8Q4Y45。",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000042",
"attribute": "常住城市",
"answerable": false,
"expected": "VAL-VA2XR4ND",
"reply": "您的默认语言是 VAL-W4NLC6Y6,您的出生城市是 VAL-6AVTPXZE,您的课程名称是 VAL-59JAQXQE,您的起床时间是 VAL-W56V6ELE",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000022",
"attribute": "运动习惯",
"answerable": false,
"expected": "VAL-7RG27UUU",
"reply": "关于您服务的对象,当前长期记忆中未包含相关信息,无法回答。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000077",
"attribute": "咖啡口味",
"answerable": false,
"expected": "VAL-MH645VGQ",
"reply": "您随身那台设备的末端数字是 VAL-N9RQGGEM。",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 1
},
{
"id": "zero-overlap-000186",
"attribute": "阅读工具",
"answerable": false,
"expected": "VAL-LSSKXTKX",
"reply": "关于您最近报名的课程内容,当前长期记忆中未包含相关证据,无法回答。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000095",
"attribute": "团队名称",
"answerable": false,
"expected": "VAL-9XSGFFKT",
"reply": "您的默认语言是 VAL-LBW9VEU5,课程名称是 VAL-DEU57W65,出生城市是 VAL-3T7B5W22,起床时间是 VAL-9BGXWX2",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000200",
"attribute": "手机尾号",
"answerable": false,
"expected": "VAL-87U9573K",
"reply": "关于您使用的机器档位,当前长期记忆中未包含此信息,证据不足,无法回答。已知事实仅涉及您的常用编辑器:user;常用编辑器;VAL-T8QM27AS。",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 1
},
{
"id": "zero-overlap-000134",
"attribute": "运动习惯",
"answerable": false,
"expected": "VAL-K2XP56JM",
"reply": "您的团队名称是 VAL-TYFQ7RE2。user;团队名称;VAL-TYFQ7RE2",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000156",
"attribute": "手机尾号",
"answerable": false,
"expected": "VAL-LT5AGPC2",
"reply": "您的紧急联系人姓氏是 VAL-LGUH8DRR,团队名称是 VAL-5PR4ZXGX,客户名称是 VAL-JHNXGADZ,阅读工具是 VAL-UQXK9AZ7,课程",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000014",
"attribute": "课程名称",
"answerable": false,
"expected": "VAL-BYHPZURM",
"reply": "您的团队名称是 user;团队名称;VAL-CC9Z7XVY。",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000049",
"attribute": "午餐偏好",
"answerable": false,
"expected": "VAL-2SW8J6P7",
"reply": "关于您所在的小组信息,当前长期记忆中未包含相关记录。已知信息包括:user;课程名称;VAL-RCSW5FHA;user;紧急联系人姓氏;VAL-JTC9REXD;user;宿舍",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000233",
"attribute": "默认语言",
"answerable": false,
"expected": "VAL-7Z7GKSVP",
"reply": "您的团队名称是 VAL-ZEQ9YFZ7。",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000133",
"attribute": "紧急联系人姓氏",
"answerable": false,
"expected": "VAL-7L7Q38BR",
"reply": "您的白天上班地点(工位楼层)是 VAL-UY99NYLA。",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000289",
"attribute": "通勤方式",
"answerable": false,
"expected": "VAL-MPCP37YU",
"reply": "关于您翻书时依赖的具体习惯或偏好,当前长期记忆中未包含相关记录。已知信息包括:user;课程名称;VAL-BB9VQBPD;user;运动习惯;VAL-VDC24N9",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000251",
"attribute": "入职年份",
"answerable": false,
"expected": "VAL-MZAMG4PA",
"reply": "关于您翻书时依赖的具体习惯或偏好,当前长期记忆中未包含相关记录。已知信息包括:user;课程名称;VAL-6XNPRKHU;运动习惯;VAL-SU7SPKUL;",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000272",
"attribute": "手机尾号",
"answerable": false,
"expected": "VAL-G6R4SSAR",
"reply": "我的项目代号是 VAL-YKGHW6B2。我的阅读工具是 VAL-28ZAPDW7。我的课程名称是 VAL-GKMBTT38。我的团队名称是 VAL-5SH59B",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000162",
"attribute": "运动习惯",
"answerable": false,
"expected": "VAL-XX4BJ68R",
"reply": "根据长期记忆证据,您的关键联系信息如下:\n- 默认语言:VAL-T6Z8A8Y9\n- 课程名称:VAL-RCG6KT7B\n- 出生城市:VAL-V",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000222",
"attribute": "档案标识",
"answerable": false,
"expected": "VAL-QC9CSDPT",
"reply": "您的交流语言未在当前长期记忆中记录,无法回答。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000020",
"attribute": "运动习惯",
"answerable": false,
"expected": "VAL-4WR8CJPQ",
"reply": "关于您坐在哪一间房的信息,当前长期记忆中未包含此数据,无法回答。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000039",
"attribute": "团队名称",
"answerable": false,
"expected": "VAL-3PQ7DMM4",
"reply": "关于您翻书时依赖的具体习惯或偏好,当前长期记忆中未包含相关记录。已知信息包括:user;课程名称;VAL-WD9K3FT3;user;运动习惯;VAL-JUZ426",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000066",
"attribute": "宿舍楼号",
"answerable": false,
"expected": "VAL-F9CBCKRA",
"reply": "收信时您的地址后半段是 VAL-ACS4FE7W。\n其他已保存信息如下:\n- 出生城市:VAL-ACS4FE7W\n- 宿舍楼号:VAL-F9CBCK",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000292",
"attribute": "办公城市",
"answerable": false,
"expected": "VAL-Z57S5YFH",
"reply": "收信时您的地址后半段是 VAL-BSR6K6QU。",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000094",
"attribute": "设备型号",
"answerable": false,
"expected": "VAL-WNVEZKXA",
"reply": "您的默认语言是 VAL-GELDDSM7,课程名称是 VAL-859DKJCX,出生城市是 VAL-YXYKCVJ8,起床时间是 VAL-CAVECGSD,午餐偏好是 VAL-",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000166",
"attribute": "运动习惯",
"answerable": false,
"expected": "VAL-J453FW7H",
"reply": "关于您何时开始在这家公司上班,当前记忆中没有相关记录,无法提供确切答案。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 1
},
{
"id": "zero-overlap-000249",
"attribute": "团队名称",
"answerable": false,
"expected": "VAL-8MJTBPJX",
"reply": "收信时地址的后半段是 VAL-VM6X8APT。user;常住城市;VAL-VM6X8APT",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000131",
"attribute": "工位楼层",
"answerable": false,
"expected": "VAL-WA8VB5MZ",
"reply": "根据长期记忆证据,您的默认语言是 VAL-5E9TSP8M,出生城市是 VAL-B6YYHYN5,起床时间是 VAL-E4APPRTR,课程名称是 VAL-9FLEK",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000029",
"attribute": "项目代号",
"answerable": false,
"expected": "VAL-JKN8YNS8",
"reply": "我的工位楼层是 VAL-DL6VUFV8。",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000286",
"attribute": "入职年份",
"answerable": false,
"expected": "VAL-HZ3H82PN",
"reply": "关于您的闹钟设置时间,当前长期记忆中未包含该信息。已知信息如下:user;设备型号;VAL-2VHYRYJP;入职年份;VAL-HCW4XTZP;常住城市;VAL-XUV",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000264",
"attribute": "紧急联系人姓氏",
"answerable": false,
"expected": "VAL-HJX2VWU7",
"reply": "关于您每天如何到公司,当前长期记忆中未包含交通方式或通勤路线的相关证据。\nuser;办公城市;VAL-XP2M992C\nuser;常住城市;VAL-WRER4CPX",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000224",
"attribute": "阅读工具",
"answerable": false,
"expected": "VAL-CP9SFPRU",
"reply": "关于您使用的机器档位,当前长期记忆中未包含此信息,证据不足,无法回答。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 1
},
{
"id": "zero-overlap-000254",
"attribute": "紧急联系人姓氏",
"answerable": false,
"expected": "VAL-YA4DYD4T",
"reply": "关于您何时开始在这家公司上班,当前记忆中没有相关记录。已知信息:user;办公城市;VAL-T7WB8GDW。",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 1
},
{
"id": "zero-overlap-000291",
"attribute": "常住城市",
"answerable": false,
"expected": "VAL-36J895QM",
"reply": "关于您的上级信息,当前长期记忆中未包含相关记录。已知信息包括:user;紧急联系人姓氏;VAL-9SAUTM24;客户名称;VAL-AMK2PGQX;默认语言;VAL",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000287",
"attribute": "课程名称",
"answerable": false,
"expected": "VAL-9FVF5F7K",
"reply": "关于您翻书时依赖的具体习惯或偏好,当前长期记忆中未包含相关记录。已知信息包括:user;课程名称;VAL-AGGV7KJD;运动习惯;VAL-NAAS25FZ;",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000167",
"attribute": "常住城市",
"answerable": false,
"expected": "VAL-E5HUQXKW",
"reply": "关于您睡觉的地儿是第几排,当前长期记忆中未包含此信息,无法回答。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000058",
"attribute": "咖啡口味",
"answerable": false,
"expected": "VAL-A86MFKEH",
"reply": "关于您翻书时依赖的具体习惯或偏好,当前长期记忆中未包含相关信息。已知信息如下:user;翻书依赖;未知。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
},
{
"id": "zero-overlap-000151",
"attribute": "运动习惯",
"answerable": false,
"expected": "VAL-PHKP82V7",
"reply": "关于您小时候待的地方,当前长期记忆中未包含此信息。已知信息如下:user;常住城市;VAL-Q9M7JUXB;办公城市;VAL-MRKQZ6JF;宿舍楼号;",
"correct": false,
"wrong_attribute": false,
"leaked": true,
"records_written": 24,
"records_active": 23,
"stop_reason": "evidence_found",
"records_selected": 8
}
]
}
}