wpy/natural-memory-nm21
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
natural-memory-nm21/nm2_1_fix_C_runtime_e2e.json
T
WpyQwq 643e22ecb9 Natural Memory NM2.1: 记忆路由器分叉、数据集缺陷修复与全轴评测证据 
- 引入 MemoryRouterXL 与 v5/v6 流式多线程训练/编码管线
- 修复 prepare_memory_router_dataset 候选池重建缺陷（mega 家族 3568x 加速，输出逐字节相同）
- 修复 v5 被破坏的拒答与多跳标签（train 未知样本 319 -> 16319，multi_hop 平均正例 1.00 -> 2.00）
- 同存储预算下 V2-128 v6 逐轴 22/22 通过：Top-1 41.12% -> 94.62%，未知拒答 0.00% -> 100.00%
- 记录三条被实测推翻的显然优化（logits_to_keep=1 反而慢 55%、XL 容量未带来收益）
- 记忆手术跨架构可移植性 14/14，读写关闭时与原生模型逐位相同
2026-09-19 11:11:31 +08:00

813 lines
29 KiB
JSON
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
{
"eval_file": "data/zero_overlap/eval.jsonl",
"episodes": 48,
"candidate_count": 24,
"router": "checkpoints\\router_replay_v7_v2_128\\memory_router_v2.pt",
"results": {
"prior=1.00 blend=0.50": {
"router": "prior=1.00 blend=0.50",
"cases": 48,
"answerable_cases": 40,
"unknown_cases": 8,
"accuracy_pct": 70.0,
"wrong_attribute_pct": 27.5,
"unknown_leak_pct": 0.0,
"mean_records_selected": 5.208333333333333,
"records_active_min": 23,
"records_active_max": 24,
"stop_reasons": {
"evidence_found": 40,
"attribute_not_covered": 8
},
"top_score_answerable": {
"n": 40,
"min": 0.84,
"p10": 1.19,
"p50": 2.73,
"p90": 8.52,
"max": 8.52
},
"top_score_unknown": {
"n": 8,
"min": 0.0,
"p10": 0.0,
"p50": 0.0,
"p90": 0.0,
"max": 0.0
},
"seconds": 288.5
}
},
"rows": {
"prior=1.00 blend=0.50": [
{
"id": "zero-overlap-000259",
"attribute": "出生城市",
"answerable": true,
"expected": "VAL-M9VSJAEX",
"reply": "您的出生城市是 VAL-M9VSJAEX。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 2.313681895152214,
"need_memory": true
},
{
"id": "zero-overlap-000219",
"attribute": "咖啡口味",
"answerable": true,
"expected": "VAL-M3W7MRJR",
"reply": "关于您早上那杯咖啡的豆子偏好，当前记忆中没有相关记录。已知信息仅包含：user；午餐偏好；VAL-Y4Z5H2FG。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.515033109300843,
"need_memory": true
},
{
"id": "zero-overlap-000193",
"attribute": "午餐偏好",
"answerable": true,
"expected": "VAL-HG84H5TD",
"reply": "您的午餐偏好是 user；午餐偏好；VAL-HG84H5TD。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 0.953831069469452,
"need_memory": true
},
{
"id": "zero-overlap-000080",
"attribute": "入职年份",
"answerable": true,
"expected": "VAL-GFT4HRX2",
"reply": "关于您何时开始在这家公司上班，当前记忆中没有相关记录。已知信息：user；办公城市；VAL-TCNMN9HW。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.51429933919514,
"need_memory": true
},
{
"id": "zero-overlap-000285",
"attribute": "起床时间",
"answerable": true,
"expected": "VAL-4K3RQQKF",
"reply": "我的闹钟设定时间是 VAL-4K3RQQKF。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 3.2757300050727958,
"need_memory": true
},
{
"id": "zero-overlap-000246",
"attribute": "常住城市",
"answerable": false,
"expected": "VAL-6BWJYDE4",
"reply": "作为一个人工智能助手，我并没有“小时候”的概念，也无法知道您具体待过哪里。\n\n不过，如果您是想询问**您自己**小时候待的地方，或者想聊聊关于某个特定地点的童年回忆，我很乐意",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 23,
"stop_reason": "attribute_not_covered",
"records_selected": 0,
"top_score": 0.0,
"need_memory": false
},
{
"id": "zero-overlap-000204",
"attribute": "阅读工具",
"answerable": true,
"expected": "VAL-ZRLSPU5V",
"reply": "我翻书时依赖的长期记忆中没有关于“翻书”习惯的记录，因此无法回答您依赖什么。",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 2.989592991362775,
"need_memory": true
},
{
"id": "zero-overlap-000078",
"attribute": "常住城市",
"answerable": true,
"expected": "VAL-TQBY65U8",
"reply": "您的常住城市是 VAL-TQBY65U8。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 1.4848385143280027,
"need_memory": true
},
{
"id": "zero-overlap-000104",
"attribute": "午餐偏好",
"answerable": true,
"expected": "VAL-SRFC842D",
"reply": "您的午餐偏好是 user；午餐偏好；VAL-SRFC842D。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 1.1867529082298276,
"need_memory": true
},
{
"id": "zero-overlap-000281",
"attribute": "咖啡口味",
"answerable": true,
"expected": "VAL-3GP4STZX",
"reply": "关于您早上饮用的咖啡偏好哪种豆子，当前记忆中没有相关记录。已知信息仅包含：user；午餐偏好；VAL-XNN7T8EK。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.515033109300843,
"need_memory": true
},
{
"id": "zero-overlap-000106",
"attribute": "咖啡口味",
"answerable": false,
"expected": "VAL-837DYYWV",
"reply": "作为一个人工智能助手，我**无法直接访问您的设备、查看您的闹钟设置或获取您的实时时间信息**。\n\n要找到您的闹钟设定时间，您可以尝试以下方法：\n\n1. **查看手机/电脑屏幕",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 23,
"stop_reason": "attribute_not_covered",
"records_selected": 0,
"top_score": 0.0,
"need_memory": false
},
{
"id": "zero-overlap-000091",
"attribute": "运动习惯",
"answerable": true,
"expected": "VAL-92MQHEDL",
"reply": "您的运动习惯是 VAL-92MQHEDL。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 0.8434395718574524,
"need_memory": true
},
{
"id": "zero-overlap-000088",
"attribute": "设备型号",
"answerable": true,
"expected": "VAL-JY6S7EYF",
"reply": "关于您使用的机器档次，当前长期记忆中未包含此信息，无法回答。已知信息仅涉及：user；常用编辑器；VAL-3KPKRM4J。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.516791512356487,
"need_memory": true
},
{
"id": "zero-overlap-000244",
"attribute": "午餐偏好",
"answerable": false,
"expected": "VAL-2VLHC26V",
"reply": "您好！作为一个人工智能助手，我并没有直接服务于特定某一家公司或机构。我的设计目标是**为所有用户**提供通用、安全、有帮助的信息和服务。\n\n不过，如果您是在某个具体平台（如企业微信",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 23,
"stop_reason": "attribute_not_covered",
"records_selected": 0,
"top_score": 0.0,
"need_memory": false
},
{
"id": "zero-overlap-000282",
"attribute": "咖啡口味",
"answerable": true,
"expected": "VAL-Q2AQQMUP",
"reply": "关于您早上那杯咖啡的豆子偏好，当前记忆中没有相关记录。已知信息仅包含：user；午餐偏好；VAL-5ATEHVSM。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.515033109300843,
"need_memory": true
},
{
"id": "zero-overlap-000056",
"attribute": "主管姓名",
"answerable": true,
"expected": "VAL-VDL8H7Y9",
"reply": "您的主管姓名是 VAL-VDL8H7Y9。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 2.049178622612938,
"need_memory": true
},
{
"id": "zero-overlap-000184",
"attribute": "项目代号",
"answerable": true,
"expected": "VAL-3H5TQEC4",
"reply": "您的项目代号是 VAL-3H5TQEC4。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 3.4711107313126073,
"need_memory": true
},
{
"id": "zero-overlap-000198",
"attribute": "手机尾号",
"answerable": true,
"expected": "VAL-7T3XVMVH",
"reply": "您随身那台设备的末端数字是 4N，该设备的完整型号为 VAL-9MWPDS4N。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.513663491707547,
"need_memory": true
},
{
"id": "zero-overlap-000258",
"attribute": "阅读工具",
"answerable": true,
"expected": "VAL-SHRY3GNG",
"reply": "我翻书时依赖的实体是 user，属性是阅读工具，已确认值为 VAL-SHRY3GNG。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 2.9794869642364628,
"need_memory": true
},
{
"id": "zero-overlap-000030",
"attribute": "课程名称",
"answerable": true,
"expected": "VAL-BQY3K5EV",
"reply": "您的课程名称是 VAL-BQY3K5EV。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 1.1944098997116086,
"need_memory": true
},
{
"id": "zero-overlap-000196",
"attribute": "手机尾号",
"answerable": true,
"expected": "VAL-ZHN2NMZV",
"reply": "您随身那台设备的末端数字是 W9，该设备的完整型号为 VAL-5VBJ22W9。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.513663491707547,
"need_memory": true
},
{
"id": "zero-overlap-000268",
"attribute": "宿舍楼号",
"answerable": true,
"expected": "VAL-QVYMWTLK",
"reply": "关于您睡觉的地儿是第几排，当前长期记忆中未包含此信息。已知信息如下：user；宿舍楼号；VAL-QVYMWTLK；常住城市；VAL-NW2E2ZWE；",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 2.7256882035478824,
"need_memory": true
},
{
"id": "zero-overlap-000006",
"attribute": "档案标识",
"answerable": true,
"expected": "VAL-8EFN3KGR",
"reply": "您的档案标识是 VAL-8EFN3KGR。user；档案标识；VAL-8EFN3KGR",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 3.7116483740847537,
"need_memory": true
},
{
"id": "zero-overlap-000047",
"attribute": "咖啡口味",
"answerable": true,
"expected": "VAL-PW46K9K9",
"reply": "关于您早上饮用的咖啡偏好哪种豆子，当前记忆中没有相关记录。已知信息仅包含您的午餐偏好是 VAL-BFW4LVTV。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.515033109300843,
"need_memory": true
},
{
"id": "zero-overlap-000173",
"attribute": "客户名称",
"answerable": false,
"expected": "VAL-P84HTKZR",
"reply": "作为一个人工智能助手，我**无法知道您随身设备的型号或数字**，因为我没有访问您个人设备信息的权限，也无法看到您正在使用的具体设备。\n\n如果您指的是：\n- **手机/电脑型号**：",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 23,
"stop_reason": "attribute_not_covered",
"records_selected": 0,
"top_score": 0.0,
"need_memory": false
},
{
"id": "zero-overlap-000000",
"attribute": "出生城市",
"answerable": true,
"expected": "VAL-PHB8FNF9",
"reply": "您的出生城市是 VAL-PHB8FNF9。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 2.455323672186914,
"need_memory": true
},
{
"id": "zero-overlap-000266",
"attribute": "出生城市",
"answerable": true,
"expected": "VAL-K7FT5XFB",
"reply": "您的出生城市是 VAL-K7FT5XFB。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 2.1730912274276046,
"need_memory": true
},
{
"id": "zero-overlap-000051",
"attribute": "项目代号",
"answerable": false,
"expected": "VAL-KG4RZYH5",
"reply": "作为一个人工智能助手，我并没有实体，也没有“待得最久”的物理空间或地理位置。我的“存在”体现在每一次与您的对话中。\n\n不过，如果您是在寻找**您自己**待得最久的地方",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 23,
"stop_reason": "attribute_not_covered",
"records_selected": 0,
"top_score": 0.0,
"need_memory": false
},
{
"id": "zero-overlap-000061",
"attribute": "常住城市",
"answerable": true,
"expected": "VAL-3SUNPNRD",
"reply": "您的常住城市是 VAL-3SUNPNRD，办公城市是 VAL-XYRY8BUQ，出生城市是 VAL-GE828D22，通勤方式是 VAL-2CXTE3TB，入职年份",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 1.6610547590255735,
"need_memory": true
},
{
"id": "zero-overlap-000214",
"attribute": "客户名称",
"answerable": true,
"expected": "VAL-MV382EEF",
"reply": "您的服务对象是 VAL-MV382EEF。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 1.2588214325904845,
"need_memory": true
},
{
"id": "zero-overlap-000092",
"attribute": "宿舍楼号",
"answerable": true,
"expected": "VAL-PLZZ659B",
"reply": "关于您睡觉的地儿是第几排，当前长期记忆中未包含此信息。已知信息如下：user；宿舍楼号；VAL-PLZZ659B。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 2.809071942542481,
"need_memory": true
},
{
"id": "zero-overlap-000140",
"attribute": "宿舍楼号",
"answerable": true,
"expected": "VAL-RMDSB297",
"reply": "关于您睡觉的地儿是第几排，当前长期记忆中未包含此信息。已知信息如下：user；宿舍楼号；VAL-RMDSB297。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 2.8702512824281925,
"need_memory": true
},
{
"id": "zero-overlap-000164",
"attribute": "客户名称",
"answerable": true,
"expected": "VAL-BB55YHW5",
"reply": "您的客户名称是 VAL-BB55YHW5。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 1.2650055932998656,
"need_memory": true
},
{
"id": "zero-overlap-000213",
"attribute": "通勤方式",
"answerable": true,
"expected": "VAL-M58DDQX2",
"reply": "您的通勤方式是 VAL-M58DDQX2。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 2.0363681795533712,
"need_memory": true
},
{
"id": "zero-overlap-000122",
"attribute": "通勤方式",
"answerable": true,
"expected": "VAL-VULSNGJ3",
"reply": "您的通勤方式是 VAL-VULSNGJ3。user；通勤方式；VAL-VULSNGJ3",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 2.0330638098057325,
"need_memory": true
},
{
"id": "zero-overlap-000168",
"attribute": "常住城市",
"answerable": true,
"expected": "VAL-8TQ8737C",
"reply": "您的常住城市是 VAL-8TQ8737C。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 1.4606248426437376,
"need_memory": true
},
{
"id": "zero-overlap-000175",
"attribute": "客户名称",
"answerable": true,
"expected": "VAL-QJNRRE8X",
"reply": "您的客户名称是 VAL-QJNRRE8X。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 1.2628431367874144,
"need_memory": true
},
{
"id": "zero-overlap-000144",
"attribute": "入职年份",
"answerable": false,
"expected": "VAL-GLHQMRN6",
"reply": "作为一个人工智能助手，我**无法知道您睡觉的具体位置**，因为我无法感知现实世界中的物理环境，也无法获取您的个人生活信息。\n\n如果您是在询问某个特定场所（如学校教室、电影院、酒店或",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 23,
"stop_reason": "attribute_not_covered",
"records_selected": 0,
"top_score": 0.0,
"need_memory": false
},
{
"id": "zero-overlap-000002",
"attribute": "午餐偏好",
"answerable": true,
"expected": "VAL-HEE2QMH2",
"reply": "您的午餐偏好是 user；午餐偏好；VAL-HEE2QMH2。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 0.9400680589675904,
"need_memory": true
},
{
"id": "zero-overlap-000171",
"attribute": "常住城市",
"answerable": true,
"expected": "VAL-KZA246K9",
"reply": "您的常住城市是 VAL-KZA246K9，办公城市是 VAL-Z8QBE5HQ，出生城市是 VAL-N8SWTE6N，宿舍楼号是 VAL-XZY4WPYH，通勤",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 1.49963095664978,
"need_memory": true
},
{
"id": "zero-overlap-000154",
"attribute": "入职年份",
"answerable": true,
"expected": "VAL-7Z9CBUX8",
"reply": "关于您何时开始在这家公司上班，当前记忆中没有相关记录。已知信息：user；办公城市；VAL-BP7253WL。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.51429933919514,
"need_memory": true
},
{
"id": "zero-overlap-000234",
"attribute": "工位楼层",
"answerable": true,
"expected": "VAL-87M7HVC7",
"reply": "您的工位楼层是 VAL-87M7HVC7。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 3.1934859675651266,
"need_memory": true
},
{
"id": "zero-overlap-000083",
"attribute": "办公城市",
"answerable": true,
"expected": "VAL-YBVYV97X",
"reply": "您的办公城市是 VAL-YBVYV97X。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 2.3717270611639645,
"need_memory": true
},
{
"id": "zero-overlap-000079",
"attribute": "出生城市",
"answerable": true,
"expected": "VAL-7R8WVAFG",
"reply": "您的出生城市是 VAL-7R8WVAFG。",
"correct": true,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 2.4271069115554122,
"need_memory": true
},
{
"id": "zero-overlap-000180",
"attribute": "客户名称",
"answerable": false,
"expected": "VAL-3F87H6UT",
"reply": "作为一个人工智能助手，我**无法知道您随身设备的型号或数字**，因为我没有访问您个人设备信息的权限，也无法看到您正在使用的具体设备。\n\n如果您指的是：\n- **手机/电脑型号**：",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 23,
"stop_reason": "attribute_not_covered",
"records_selected": 0,
"top_score": 0.0,
"need_memory": false
},
{
"id": "zero-overlap-000025",
"attribute": "紧急联系人姓氏",
"answerable": true,
"expected": "VAL-STECQ3PX",
"reply": "当遇到突发问题或需要联系支持时，您可以直接联系您的专属支持团队。具体联系渠道包括：\n- 默认语言：VAL-PRVTUCP7\n- 出生城市：VAL-KYBRBBKV",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 8,
"top_score": 8.516015553315961,
"need_memory": true
},
{
"id": "zero-overlap-000187",
"attribute": "手机尾号",
"answerable": true,
"expected": "VAL-PXRHJ49F",
"reply": "您随身那台设备的末端数字是 H9，该设备的完整型号为 VAL-ZPUPV9H9。",
"correct": false,
"wrong_attribute": true,
"leaked": false,
"records_written": 24,
"records_active": 24,
"stop_reason": "evidence_found",
"records_selected": 1,
"top_score": 8.513663491707547,
"need_memory": true
},
{
"id": "zero-overlap-000045",
"attribute": "办公城市",
"answerable": false,
"expected": "VAL-CG4X6ZWD",
"reply": "作为一个人工智能助手，我并没有实体，也没有“待得最久”的物理空间或地理位置。我的“存在”体现在每一次与您的对话中。\n\n不过，如果您是在寻找**您自己**待得最久的地方",
"correct": false,
"wrong_attribute": false,
"leaked": false,
"records_written": 24,
"records_active": 23,
"stop_reason": "attribute_not_covered",
"records_selected": 0,
"top_score": 0.0,
"need_memory": false
}
]
}
}
Reference in New Issue View Git Blame Copy Permalink