Initial commit: Natural Memory Agent Lab：模型服务 + TypeScript agent + 记忆能力场景评测（冲突更新、跨会话回忆、遗忘撤回、多跳、未知拒答）
This commit is contained in:
WpyQwq
@@ -0,0 +1,3 @@
|
||||
{"session":"conflict_update:off:s1","turn":0,"userMessage":"记一下：我的默认语言是 日语。","assistantText":"好的，已记录您的默认语言为日语。","toolCalls":[{"name":"notes","args":{"action":"append","text":"用户的默认语言是 日语。"}}],"toolResults":[{"name":"notes","result":"{\"content\":[{\"type\":\"text\",\"text\":\"appended\"}],\"details\":{\"path\":\"H:\\\\Memory\\\\agent_lab\\\\agent\\\\runs\\\\suite5\\\\scratchpad\\\\off.txt\"}}"}],"wallSeconds":6.737,"diagnostics":{"turn":120,"kind":"chat","at":1789387437.3664691,"user_message":"","assistant_text":"好的，已记录您的默认语言为日语。","reasoning_chars":0,"router_decisions":[{"need_memory":false,"page_ids":[],"record_ids":[],"page_scores":[],"record_scores":[],"hop_count":0,"hop_trace":[],"confidence":0.000004011214514321182,"stop_reason":"router_abstained","top_score":0,"score_margin":0,"evidence_score":0}],"prefix_tokens":0,"read_seconds":0.7857468000001973,"coverage":{"configured":true,"loaded":true,"bound_to_bank":true,"vocabulary_fraction":0.9,"head":{"path":"H:\\Memory\\V2_dpskw\\qwen3_5_4b_natural_memory_v2_1\\memory_attribute_head.pt","attributes":24,"vocabulary":["主管姓名","入职年份","出生城市","办公城市","午餐偏好","咖啡口味","团队名称","客户名称","宿舍楼号","工位楼层","常住城市","常用编辑器","手机尾号","档案标识","紧急联系人姓氏","设备型号","课程名称","起床时间","运动习惯","通勤方式","邮箱域名","阅读工具","项目代号","默认语言"],"coverage_stats":{"applicable":0,"bypassed":120}}},"no_evidence":false,"memory_mode":"off","memory_changed":false,"reset_triggered":false,"tool_calls":[],"overrides":{},"prompt_tokens":1867,"completion_tokens":13,"generation_seconds":2.132,"total_seconds":2.138,"snapshot":{"router_decisions":[{"need_memory":false,"page_ids":[],"record_ids":[],"page_scores":[],"record_scores":[],"hop_count":0,"hop_trace":[],"confidence":0.000004011214514321182,"stop_reason":"router_abstained","top_score":0,"score_margin":0,"evidence_score":0}],"no_evidence":false,"prefix_tokens":0,"prefix_used":false,"read_seconds":0.7857468000001973,"auto_memory_probability":null,"auto_forget_probability":[0.004185875877737999],"context_compaction":{"compacted":false,"archived_records":0,"archived_tokens":0,"original_tokens":[1867],"retained_tokens":[1867],"hot_limit":32768,"archive_requested":true},"coverage":{"configured":true,"loaded":true,"bound_to_bank":true,"vocabulary_fraction":0.9,"head":{"path":"H:\\Memory\\V2_dpskw\\qwen3_5_4b_natural_memory_v2_1\\memory_attribute_head.pt","attributes":24,"vocabulary":["主管姓名","入职年份","出生城市","办公城市","午餐偏好","咖啡口味","团队名称","客户名称","宿舍楼号","工位楼层","常住城市","常用编辑器","手机尾号","档案标识","紧急联系人姓氏","设备型号","课程名称","起床时间","运动习惯","通勤方式","邮箱域名","阅读工具","项目代号","默认语言"],"coverage_stats":{"applicable":0,"bypassed":120}}},"memory":{"pages":0,"max_pages":32768,"page_capacity":32,"capacity_records":1048576,"open_pages":0,"records":0,"active_records":0,"superseded_records":0,"retracted_records":0,"quarantined_records":0,"hot_pages":0,"warm_pages":0,"cold_pages":0,"active_conflict_keys":0,"coarse_index_buckets":0,"last_coarse_candidates":0,"storage_mode":"embedded","resident_records":0,"resident_pages":0,"gpu_cache_records":0,"gpu_cache_tokens":0,"gpu_cache_bytes":0,"gpu_cache_hits":0,"gpu_cache_misses":0,"gpu_cache_fallbacks":0,"gpu_cache_alloc_failures":0,"gpu_cache_device":"cuda:0","gpu_cache_limit_records":256,"gpu_cache_limit_tokens":131072,"gpu_cache_reserve_mb":2048,"gpu_cache_adaptive":true,"gpu_cache_last_free_mb":null,"kv_budget":{"max_tokens":32768,"trigger_tokens":29491},"enabled":true,"router_ready":true}}},"memoryAfter":{"pages":0,"max_pages":32768,"page_capacity":32,"capacity_records":1048576,"open_pages":0,"records":0,"active_records":0,"superseded_records":0,"retracted_records":0,"quarantined_records":0,"hot_pages":0,"warm_pages":0,"cold_pages":0,"active_conflict_keys":0,"coarse_index_buckets":0,"last_coarse_candidates":0,"storage_mode":"embedded","resident_records":0,"resident_pages":0,"gpu_cache_records":0,"gpu_cache_tokens":0,"gpu_cache_bytes":0,"gpu_cache_hits":0,"gpu_cache_misses":0,"gpu_cache_fallbacks":0,"gpu_cache_alloc_failures":0,"gpu_cache_device":"cuda:0","gpu_cache_limit_records":256,"gpu_cache_limit_tokens":131072,"gpu_cache_reserve_mb":2048,"gpu_cache_adaptive":true,"gpu_cache_last_free_mb":null,"kv_budget":{"max_tokens":32768,"trigger_tokens":29491},"enabled":true,"router_ready":true}}
|
||||
{"session":"conflict_update:off:s1","turn":1,"userMessage":"更正一下：我的默认语言改成 葡萄牙语，日语作废。","assistantText":"好的，已更正您的默认语言为葡萄牙语，日语已作废。","toolCalls":[{"name":"notes","args":{"action":"append","text":"用户的默认语言是 葡萄牙语。日语作废。"}}],"toolResults":[{"name":"notes","result":"{\"content\":[{\"type\":\"text\",\"text\":\"appended\"}],\"details\":{\"path\":\"H:\\\\Memory\\\\agent_lab\\\\agent\\\\runs\\\\suite5\\\\scratchpad\\\\off.txt\"}}"}],"wallSeconds":10.213,"diagnostics":{"turn":122,"kind":"chat","at":1789387447.5819647,"user_message":"","assistant_text":"好的，已更正您的默认语言为葡萄牙语，日语已作废。","reasoning_chars":0,"router_decisions":[{"need_memory":false,"page_ids":[],"record_ids":[],"page_scores":[],"record_scores":[],"hop_count":0,"hop_trace":[],"confidence":7.154267791520397e-7,"stop_reason":"router_abstained","top_score":0,"score_margin":0,"evidence_score":0}],"prefix_tokens":0,"read_seconds":1.385789200000545,"coverage":{"configured":true,"loaded":true,"bound_to_bank":true,"vocabulary_fraction":0.9,"head":{"path":"H:\\Memory\\V2_dpskw\\qwen3_5_4b_natural_memory_v2_1\\memory_attribute_head.pt","attributes":24,"vocabulary":["主管姓名","入职年份","出生城市","办公城市","午餐偏好","咖啡口味","团队名称","客户名称","宿舍楼号","工位楼层","常住城市","常用编辑器","手机尾号","档案标识","紧急联系人姓氏","设备型号","课程名称","起床时间","运动习惯","通勤方式","邮箱域名","阅读工具","项目代号","默认语言"],"coverage_stats":{"applicable":0,"bypassed":122}}},"no_evidence":false,"memory_mode":"off","memory_changed":false,"reset_triggered":false,"tool_calls":[],"overrides":{},"prompt_tokens":1981,"completion_tokens":18,"generation_seconds":3.856,"total_seconds":3.862,"snapshot":{"router_decisions":[{"need_memory":false,"page_ids":[],"record_ids":[],"page_scores":[],"record_scores":[],"hop_count":0,"hop_trace":[],"confidence":7.154267791520397e-7,"stop_reason":"router_abstained","top_score":0,"score_margin":0,"evidence_score":0}],"no_evidence":false,"prefix_tokens":0,"prefix_used":false,"read_seconds":1.385789200000545,"auto_memory_probability":null,"auto_forget_probability":[0.004185875877737999],"context_compaction":{"compacted":false,"archived_records":0,"archived_tokens":0,"original_tokens":[1981],"retained_tokens":[1981],"hot_limit":32768,"archive_requested":true},"coverage":{"configured":true,"loaded":true,"bound_to_bank":true,"vocabulary_fraction":0.9,"head":{"path":"H:\\Memory\\V2_dpskw\\qwen3_5_4b_natural_memory_v2_1\\memory_attribute_head.pt","attributes":24,"vocabulary":["主管姓名","入职年份","出生城市","办公城市","午餐偏好","咖啡口味","团队名称","客户名称","宿舍楼号","工位楼层","常住城市","常用编辑器","手机尾号","档案标识","紧急联系人姓氏","设备型号","课程名称","起床时间","运动习惯","通勤方式","邮箱域名","阅读工具","项目代号","默认语言"],"coverage_stats":{"applicable":0,"bypassed":122}}},"memory":{"pages":0,"max_pages":32768,"page_capacity":32,"capacity_records":1048576,"open_pages":0,"records":0,"active_records":0,"superseded_records":0,"retracted_records":0,"quarantined_records":0,"hot_pages":0,"warm_pages":0,"cold_pages":0,"active_conflict_keys":0,"coarse_index_buckets":0,"last_coarse_candidates":0,"storage_mode":"embedded","resident_records":0,"resident_pages":0,"gpu_cache_records":0,"gpu_cache_tokens":0,"gpu_cache_bytes":0,"gpu_cache_hits":0,"gpu_cache_misses":0,"gpu_cache_fallbacks":0,"gpu_cache_alloc_failures":0,"gpu_cache_device":"cuda:0","gpu_cache_limit_records":256,"gpu_cache_limit_tokens":131072,"gpu_cache_reserve_mb":2048,"gpu_cache_adaptive":true,"gpu_cache_last_free_mb":null,"kv_budget":{"max_tokens":32768,"trigger_tokens":29491},"enabled":true,"router_ready":true}}},"memoryAfter":{"pages":0,"max_pages":32768,"page_capacity":32,"capacity_records":1048576,"open_pages":0,"records":0,"active_records":0,"superseded_records":0,"retracted_records":0,"quarantined_records":0,"hot_pages":0,"warm_pages":0,"cold_pages":0,"active_conflict_keys":0,"coarse_index_buckets":0,"last_coarse_candidates":0,"storage_mode":"embedded","resident_records":0,"resident_pages":0,"gpu_cache_records":0,"gpu_cache_tokens":0,"gpu_cache_bytes":0,"gpu_cache_hits":0,"gpu_cache_misses":0,"gpu_cache_fallbacks":0,"gpu_cache_alloc_failures":0,"gpu_cache_device":"cuda:0","gpu_cache_limit_records":256,"gpu_cache_limit_tokens":131072,"gpu_cache_reserve_mb":2048,"gpu_cache_adaptive":true,"gpu_cache_last_free_mb":null,"kv_budget":{"max_tokens":32768,"trigger_tokens":29491},"enabled":true,"router_ready":true}}
|
||||
{"session":"conflict_update:off:s2","turn":0,"userMessage":"我的默认语言是什么？只回答语言名。","assistantText":"中文","toolCalls":[],"toolResults":[],"wallSeconds":1.518,"diagnostics":{"turn":123,"kind":"chat","at":1789387449.1066668,"user_message":"","assistant_text":"中文","reasoning_chars":0,"router_decisions":[{"need_memory":false,"page_ids":[],"record_ids":[],"page_scores":[],"record_scores":[],"hop_count":0,"hop_trace":[],"confidence":0.0000041284320104750805,"stop_reason":"router_abstained","top_score":0,"score_margin":0,"evidence_score":0}],"prefix_tokens":0,"read_seconds":0.712273400000413,"coverage":{"configured":true,"loaded":true,"bound_to_bank":true,"vocabulary_fraction":0.9,"head":{"path":"H:\\Memory\\V2_dpskw\\qwen3_5_4b_natural_memory_v2_1\\memory_attribute_head.pt","attributes":24,"vocabulary":["主管姓名","入职年份","出生城市","办公城市","午餐偏好","咖啡口味","团队名称","客户名称","宿舍楼号","工位楼层","常住城市","常用编辑器","手机尾号","档案标识","紧急联系人姓氏","设备型号","课程名称","起床时间","运动习惯","通勤方式","邮箱域名","阅读工具","项目代号","默认语言"],"coverage_stats":{"applicable":0,"bypassed":123}}},"no_evidence":false,"memory_mode":"off","memory_changed":false,"reset_triggered":false,"tool_calls":[],"overrides":{},"prompt_tokens":1805,"completion_tokens":4,"generation_seconds":1.503,"total_seconds":1.514,"snapshot":{"router_decisions":[{"need_memory":false,"page_ids":[],"record_ids":[],"page_scores":[],"record_scores":[],"hop_count":0,"hop_trace":[],"confidence":0.0000041284320104750805,"stop_reason":"router_abstained","top_score":0,"score_margin":0,"evidence_score":0}],"no_evidence":false,"prefix_tokens":0,"prefix_used":false,"read_seconds":0.712273400000413,"auto_memory_probability":null,"auto_forget_probability":[0.004185875877737999],"context_compaction":{"compacted":false,"archived_records":0,"archived_tokens":0,"original_tokens":[1805],"retained_tokens":[1805],"hot_limit":32768,"archive_requested":true},"coverage":{"configured":true,"loaded":true,"bound_to_bank":true,"vocabulary_fraction":0.9,"head":{"path":"H:\\Memory\\V2_dpskw\\qwen3_5_4b_natural_memory_v2_1\\memory_attribute_head.pt","attributes":24,"vocabulary":["主管姓名","入职年份","出生城市","办公城市","午餐偏好","咖啡口味","团队名称","客户名称","宿舍楼号","工位楼层","常住城市","常用编辑器","手机尾号","档案标识","紧急联系人姓氏","设备型号","课程名称","起床时间","运动习惯","通勤方式","邮箱域名","阅读工具","项目代号","默认语言"],"coverage_stats":{"applicable":0,"bypassed":123}}},"memory":{"pages":0,"max_pages":32768,"page_capacity":32,"capacity_records":1048576,"open_pages":0,"records":0,"active_records":0,"superseded_records":0,"retracted_records":0,"quarantined_records":0,"hot_pages":0,"warm_pages":0,"cold_pages":0,"active_conflict_keys":0,"coarse_index_buckets":0,"last_coarse_candidates":0,"storage_mode":"embedded","resident_records":0,"resident_pages":0,"gpu_cache_records":0,"gpu_cache_tokens":0,"gpu_cache_bytes":0,"gpu_cache_hits":0,"gpu_cache_misses":0,"gpu_cache_fallbacks":0,"gpu_cache_alloc_failures":0,"gpu_cache_device":"cuda:0","gpu_cache_limit_records":256,"gpu_cache_limit_tokens":131072,"gpu_cache_reserve_mb":2048,"gpu_cache_adaptive":true,"gpu_cache_last_free_mb":null,"kv_budget":{"max_tokens":32768,"trigger_tokens":29491},"enabled":true,"router_ready":true}}},"memoryAfter":{"pages":0,"max_pages":32768,"page_capacity":32,"capacity_records":1048576,"open_pages":0,"records":0,"active_records":0,"superseded_records":0,"retracted_records":0,"quarantined_records":0,"hot_pages":0,"warm_pages":0,"cold_pages":0,"active_conflict_keys":0,"coarse_index_buckets":0,"last_coarse_candidates":0,"storage_mode":"embedded","resident_records":0,"resident_pages":0,"gpu_cache_records":0,"gpu_cache_tokens":0,"gpu_cache_bytes":0,"gpu_cache_hits":0,"gpu_cache_misses":0,"gpu_cache_fallbacks":0,"gpu_cache_alloc_failures":0,"gpu_cache_device":"cuda:0","gpu_cache_limit_records":256,"gpu_cache_limit_tokens":131072,"gpu_cache_reserve_mb":2048,"gpu_cache_adaptive":true,"gpu_cache_last_free_mb":null,"kv_budget":{"max_tokens":32768,"trigger_tokens":29491},"enabled":true,"router_ready":true}}
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,4 @@
|
||||
{"session":"cross_session_recall:off:s1","turn":0,"userMessage":"记一下：我的项目代号是 蓝鲸-47，部署区域是 ap-southeast-3。","assistantText":"已记录：项目代号 蓝鲸-47，部署区域 ap-southeast-3。","toolCalls":[{"name":"notes","args":{"action":"append","text":"项目代号：蓝鲸-47，部署区域：ap-southeast-3"}}],"toolResults":[{"name":"notes","result":"{\"content\":[{\"type\":\"text\",\"text\":\"appended\"}],\"details\":{\"path\":\"H:\\\\Memory\\\\agent_lab\\\\agent\\\\runs\\\\suite5\\\\scratchpad\\\\off.txt\"}}"}],"wallSeconds":7.389,"diagnostics":{"turn":131,"kind":"chat","at":1789387491.3257182,"user_message":"","assistant_text":"已记录：项目代号 蓝鲸-47，部署区域 ap-southeast-3。","reasoning_chars":0,"router_decisions":[{"need_memory":false,"page_ids":[],"record_ids":[],"page_scores":[],"record_scores":[],"hop_count":0,"hop_trace":[],"confidence":0.000003590315827750601,"stop_reason":"router_abstained","top_score":0,"score_margin":0,"evidence_score":0}],"prefix_tokens":0,"read_seconds":0.7623988999994253,"coverage":{"configured":true,"loaded":true,"bound_to_bank":true,"vocabulary_fraction":0.9,"head":{"path":"H:\\Memory\\V2_dpskw\\qwen3_5_4b_natural_memory_v2_1\\memory_attribute_head.pt","attributes":24,"vocabulary":["主管姓名","入职年份","出生城市","办公城市","午餐偏好","咖啡口味","团队名称","客户名称","宿舍楼号","工位楼层","常住城市","常用编辑器","手机尾号","档案标识","紧急联系人姓氏","设备型号","课程名称","起床时间","运动习惯","通勤方式","邮箱域名","阅读工具","项目代号","默认语言"],"coverage_stats":{"applicable":0,"bypassed":131}}},"no_evidence":false,"memory_mode":"off","memory_changed":false,"reset_triggered":false,"tool_calls":[],"overrides":{},"prompt_tokens":1892,"completion_tokens":24,"generation_seconds":2.711,"total_seconds":2.717,"snapshot":{"router_decisions":[{"need_memory":false,"page_ids":[],"record_ids":[],"page_scores":[],"record_scores":[],"hop_count":0,"hop_trace":[],"confidence":0.000003590315827750601,"stop_reason":"router_abstained","top_score":0,"score_margin":0,"evidence_score":0}],"no_evidence":false,"prefix_tokens":0,"prefix_used":false,"read_seconds":0.7623988999994253,"auto_memory_probability":null,"auto_forget_probability":[0.011167768388986588],"context_compaction":{"compacted":false,"archived_records":0,"archived_tokens":0,"original_tokens":[1892],"retained_tokens":[1892],"hot_limit":32768,"archive_requested":true},"coverage":{"configured":true,"loaded":true,"bound_to_bank":true,"vocabulary_fraction":0.9,"head":{"path":"H:\\Memory\\V2_dpskw\\qwen3_5_4b_natural_memory_v2_1\\memory_attribute_head.pt","attributes":24,"vocabulary":["主管姓名","入职年份","出生城市","办公城市","午餐偏好","咖啡口味","团队名称","客户名称","宿舍楼号","工位楼层","常住城市","常用编辑器","手机尾号","档案标识","紧急联系人姓氏","设备型号","课程名称","起床时间","运动习惯","通勤方式","邮箱域名","阅读工具","项目代号","默认语言"],"coverage_stats":{"applicable":0,"bypassed":131}}},"memory":{"pages":0,"max_pages":32768,"page_capacity":32,"capacity_records":1048576,"open_pages":0,"records":0,"active_records":0,"superseded_records":0,"retracted_records":0,"quarantined_records":0,"hot_pages":0,"warm_pages":0,"cold_pages":0,"active_conflict_keys":0,"coarse_index_buckets":0,"last_coarse_candidates":0,"storage_mode":"embedded","resident_records":0,"resident_pages":0,"gpu_cache_records":0,"gpu_cache_tokens":0,"gpu_cache_bytes":0,"gpu_cache_hits":0,"gpu_cache_misses":0,"gpu_cache_fallbacks":0,"gpu_cache_alloc_failures":0,"gpu_cache_device":"cuda:0","gpu_cache_limit_records":256,"gpu_cache_limit_tokens":131072,"gpu_cache_reserve_mb":2048,"gpu_cache_adaptive":true,"gpu_cache_last_free_mb":null,"kv_budget":{"max_tokens":32768,"trigger_tokens":29491},"enabled":true,"router_ready":true}}},"memoryAfter":{"pages":0,"max_pages":32768,"page_capacity":32,"capacity_records":1048576,"open_pages":0,"records":0,"active_records":0,"superseded_records":0,"retracted_records":0,"quarantined_records":0,"hot_pages":0,"warm_pages":0,"cold_pages":0,"active_conflict_keys":0,"coarse_index_buckets":0,"last_coarse_candidates":0,"storage_mode":"embedded","resident_records":0,"resident_pages":0,"gpu_cache_records":0,"gpu_cache_tokens":0,"gpu_cache_bytes":0,"gpu_cache_hits":0,"gpu_cache_misses":0,"gpu_cache_fallbacks":0,"gpu_cache_alloc_failures":0,"gpu_cache_device":"cuda:0","gpu_cache_limit_records":256,"gpu_cache_limit_tokens":131072,"gpu_cache_reserve_mb":2048,"gpu_cache_adaptive":true,"gpu_cache_last_free_mb":null,"kv_budget":{"max_tokens":32768,"trigger_tokens":29491},"enabled":true,"router_ready":true}}
|
||||
{"session":"cross_session_recall:off:s1","turn":1,"userMessage":"谢谢。","assistantText":"不客气！有什么需要随时告诉我。","toolCalls":[],"toolResults":[],"wallSeconds":2.16,"diagnostics":{"turn":132,"kind":"chat","at":1789387493.4889941,"user_message":"","assistant_text":"不客气！有什么需要随时告诉我。","reasoning_chars":0,"router_decisions":[{"need_memory":false,"page_ids":[],"record_ids":[],"page_scores":[],"record_scores":[],"hop_count":0,"hop_trace":[],"confidence":0.0000024636194666527444,"stop_reason":"router_abstained","top_score":0,"score_margin":0,"evidence_score":0}],"prefix_tokens":0,"read_seconds":0.8221218999997291,"coverage":{"configured":true,"loaded":true,"bound_to_bank":true,"vocabulary_fraction":0.9,"head":{"path":"H:\\Memory\\V2_dpskw\\qwen3_5_4b_natural_memory_v2_1\\memory_attribute_head.pt","attributes":24,"vocabulary":["主管姓名","入职年份","出生城市","办公城市","午餐偏好","咖啡口味","团队名称","客户名称","宿舍楼号","工位楼层","常住城市","常用编辑器","手机尾号","档案标识","紧急联系人姓氏","设备型号","课程名称","起床时间","运动习惯","通勤方式","邮箱域名","阅读工具","项目代号","默认语言"],"coverage_stats":{"applicable":0,"bypassed":132}}},"no_evidence":false,"memory_mode":"off","memory_changed":false,"reset_triggered":false,"tool_calls":[],"overrides":{},"prompt_tokens":1921,"completion_tokens":11,"generation_seconds":2.134,"total_seconds":2.156,"snapshot":{"router_decisions":[{"need_memory":false,"page_ids":[],"record_ids":[],"page_scores":[],"record_scores":[],"hop_count":0,"hop_trace":[],"confidence":0.0000024636194666527444,"stop_reason":"router_abstained","top_score":0,"score_margin":0,"evidence_score":0}],"no_evidence":false,"prefix_tokens":0,"prefix_used":false,"read_seconds":0.8221218999997291,"auto_memory_probability":null,"auto_forget_probability":[0.011167768388986588],"context_compaction":{"compacted":false,"archived_records":0,"archived_tokens":0,"original_tokens":[1921],"retained_tokens":[1921],"hot_limit":32768,"archive_requested":true},"coverage":{"configured":true,"loaded":true,"bound_to_bank":true,"vocabulary_fraction":0.9,"head":{"path":"H:\\Memory\\V2_dpskw\\qwen3_5_4b_natural_memory_v2_1\\memory_attribute_head.pt","attributes":24,"vocabulary":["主管姓名","入职年份","出生城市","办公城市","午餐偏好","咖啡口味","团队名称","客户名称","宿舍楼号","工位楼层","常住城市","常用编辑器","手机尾号","档案标识","紧急联系人姓氏","设备型号","课程名称","起床时间","运动习惯","通勤方式","邮箱域名","阅读工具","项目代号","默认语言"],"coverage_stats":{"applicable":0,"bypassed":132}}},"memory":{"pages":0,"max_pages":32768,"page_capacity":32,"capacity_records":1048576,"open_pages":0,"records":0,"active_records":0,"superseded_records":0,"retracted_records":0,"quarantined_records":0,"hot_pages":0,"warm_pages":0,"cold_pages":0,"active_conflict_keys":0,"coarse_index_buckets":0,"last_coarse_candidates":0,"storage_mode":"embedded","resident_records":0,"resident_pages":0,"gpu_cache_records":0,"gpu_cache_tokens":0,"gpu_cache_bytes":0,"gpu_cache_hits":0,"gpu_cache_misses":0,"gpu_cache_fallbacks":0,"gpu_cache_alloc_failures":0,"gpu_cache_device":"cuda:0","gpu_cache_limit_records":256,"gpu_cache_limit_tokens":131072,"gpu_cache_reserve_mb":2048,"gpu_cache_adaptive":true,"gpu_cache_last_free_mb":null,"kv_budget":{"max_tokens":32768,"trigger_tokens":29491},"enabled":true,"router_ready":true}}},"memoryAfter":{"pages":0,"max_pages":32768,"page_capacity":32,"capacity_records":1048576,"open_pages":0,"records":0,"active_records":0,"superseded_records":0,"retracted_records":0,"quarantined_records":0,"hot_pages":0,"warm_pages":0,"cold_pages":0,"active_conflict_keys":0,"coarse_index_buckets":0,"last_coarse_candidates":0,"storage_mode":"embedded","resident_records":0,"resident_pages":0,"gpu_cache_records":0,"gpu_cache_tokens":0,"gpu_cache_bytes":0,"gpu_cache_hits":0,"gpu_cache_misses":0,"gpu_cache_fallbacks":0,"gpu_cache_alloc_failures":0,"gpu_cache_device":"cuda:0","gpu_cache_limit_records":256,"gpu_cache_limit_tokens":131072,"gpu_cache_reserve_mb":2048,"gpu_cache_adaptive":true,"gpu_cache_last_free_mb":null,"kv_budget":{"max_tokens":32768,"trigger_tokens":29491},"enabled":true,"router_ready":true}}
|
||||
{"session":"cross_session_recall:off:s2","turn":0,"userMessage":"我的项目代号是什么？只回答代号本身。","assistantText":"我的项目代号是：Project Phoenix","toolCalls":[],"toolResults":[],"wallSeconds":1.845,"diagnostics":{"turn":133,"kind":"chat","at":1789387495.3334734,"user_message":"","assistant_text":"我的项目代号是：Project Phoenix","reasoning_chars":0,"router_decisions":[{"need_memory":false,"page_ids":[],"record_ids":[],"page_scores":[],"record_scores":[],"hop_count":0,"hop_trace":[],"confidence":0.000004224521035212092,"stop_reason":"router_abstained","top_score":0,"score_margin":0,"evidence_score":0}],"prefix_tokens":0,"read_seconds":0.6864501999998538,"coverage":{"configured":true,"loaded":true,"bound_to_bank":true,"vocabulary_fraction":0.9,"head":{"path":"H:\\Memory\\V2_dpskw\\qwen3_5_4b_natural_memory_v2_1\\memory_attribute_head.pt","attributes":24,"vocabulary":["主管姓名","入职年份","出生城市","办公城市","午餐偏好","咖啡口味","团队名称","客户名称","宿舍楼号","工位楼层","常住城市","常用编辑器","手机尾号","档案标识","紧急联系人姓氏","设备型号","课程名称","起床时间","运动习惯","通勤方式","邮箱域名","阅读工具","项目代号","默认语言"],"coverage_stats":{"applicable":0,"bypassed":133}}},"no_evidence":false,"memory_mode":"off","memory_changed":false,"reset_triggered":false,"tool_calls":[],"overrides":{},"prompt_tokens":1805,"completion_tokens":10,"generation_seconds":1.83,"total_seconds":1.836,"snapshot":{"router_decisions":[{"need_memory":false,"page_ids":[],"record_ids":[],"page_scores":[],"record_scores":[],"hop_count":0,"hop_trace":[],"confidence":0.000004224521035212092,"stop_reason":"router_abstained","top_score":0,"score_margin":0,"evidence_score":0}],"no_evidence":false,"prefix_tokens":0,"prefix_used":false,"read_seconds":0.6864501999998538,"auto_memory_probability":null,"auto_forget_probability":[0.011167768388986588],"context_compaction":{"compacted":false,"archived_records":0,"archived_tokens":0,"original_tokens":[1805],"retained_tokens":[1805],"hot_limit":32768,"archive_requested":true},"coverage":{"configured":true,"loaded":true,"bound_to_bank":true,"vocabulary_fraction":0.9,"head":{"path":"H:\\Memory\\V2_dpskw\\qwen3_5_4b_natural_memory_v2_1\\memory_attribute_head.pt","attributes":24,"vocabulary":["主管姓名","入职年份","出生城市","办公城市","午餐偏好","咖啡口味","团队名称","客户名称","宿舍楼号","工位楼层","常住城市","常用编辑器","手机尾号","档案标识","紧急联系人姓氏","设备型号","课程名称","起床时间","运动习惯","通勤方式","邮箱域名","阅读工具","项目代号","默认语言"],"coverage_stats":{"applicable":0,"bypassed":133}}},"memory":{"pages":0,"max_pages":32768,"page_capacity":32,"capacity_records":1048576,"open_pages":0,"records":0,"active_records":0,"superseded_records":0,"retracted_records":0,"quarantined_records":0,"hot_pages":0,"warm_pages":0,"cold_pages":0,"active_conflict_keys":0,"coarse_index_buckets":0,"last_coarse_candidates":0,"storage_mode":"embedded","resident_records":0,"resident_pages":0,"gpu_cache_records":0,"gpu_cache_tokens":0,"gpu_cache_bytes":0,"gpu_cache_hits":0,"gpu_cache_misses":0,"gpu_cache_fallbacks":0,"gpu_cache_alloc_failures":0,"gpu_cache_device":"cuda:0","gpu_cache_limit_records":256,"gpu_cache_limit_tokens":131072,"gpu_cache_reserve_mb":2048,"gpu_cache_adaptive":true,"gpu_cache_last_free_mb":null,"kv_budget":{"max_tokens":32768,"trigger_tokens":29491},"enabled":true,"router_ready":true}}},"memoryAfter":{"pages":0,"max_pages":32768,"page_capacity":32,"capacity_records":1048576,"open_pages":0,"records":0,"active_records":0,"superseded_records":0,"retracted_records":0,"quarantined_records":0,"hot_pages":0,"warm_pages":0,"cold_pages":0,"active_conflict_keys":0,"coarse_index_buckets":0,"last_coarse_candidates":0,"storage_mode":"embedded","resident_records":0,"resident_pages":0,"gpu_cache_records":0,"gpu_cache_tokens":0,"gpu_cache_bytes":0,"gpu_cache_hits":0,"gpu_cache_misses":0,"gpu_cache_fallbacks":0,"gpu_cache_alloc_failures":0,"gpu_cache_device":"cuda:0","gpu_cache_limit_records":256,"gpu_cache_limit_tokens":131072,"gpu_cache_reserve_mb":2048,"gpu_cache_adaptive":true,"gpu_cache_last_free_mb":null,"kv_budget":{"max_tokens":32768,"trigger_tokens":29491},"enabled":true,"router_ready":true}}
|
||||
{"session":"cross_session_recall:off:s3","turn":0,"userMessage":"部署区域呢？只回答区域本身。","assistantText":"未找到名为 \"deployment\" 的目录。","toolCalls":[{"name":"run_shell","args":{"command":"find / -type d -name \"deployment\" 2>/dev/null | head -5","timeout_ms":5000}}],"toolResults":[{"name":"run_shell","result":"{\"content\":[{\"type\":\"text\",\"text\":\"command failed: Command failed: pwsh -NoProfile -Command find / -type d -name \\\"deployment\\\" 2>/dev/null | head -5\\nOut-File: Could not find a part of the path 'H:\\\\dev\\\\null'.\\r\\n\"}],\"details\":{}}"}],"wallSeconds":11.977,"diagnostics":{"turn":135,"kind":"chat","at":1789387507.3257377,"user_message":"","assistant_text":"未找到名为 \"deployment\" 的目录。","reasoning_chars":0,"router_decisions":[{"need_memory":false,"page_ids":[],"record_ids":[],"page_scores":[],"record_scores":[],"hop_count":0,"hop_trace":[],"confidence":0.000001446954911443754,"stop_reason":"router_abstained","top_score":0,"score_margin":0,"evidence_score":0}],"prefix_tokens":0,"read_seconds":1.637925200000609,"coverage":{"configured":true,"loaded":true,"bound_to_bank":true,"vocabulary_fraction":0.9,"head":{"path":"H:\\Memory\\V2_dpskw\\qwen3_5_4b_natural_memory_v2_1\\memory_attribute_head.pt","attributes":24,"vocabulary":["主管姓名","入职年份","出生城市","办公城市","午餐偏好","咖啡口味","团队名称","客户名称","宿舍楼号","工位楼层","常住城市","常用编辑器","手机尾号","档案标识","紧急联系人姓氏","设备型号","课程名称","起床时间","运动习惯","通勤方式","邮箱域名","阅读工具","项目代号","默认语言"],"coverage_stats":{"applicable":0,"bypassed":135}}},"no_evidence":false,"memory_mode":"off","memory_changed":false,"reset_triggered":false,"tool_calls":[],"overrides":{},"prompt_tokens":1955,"completion_tokens":13,"generation_seconds":3.739,"total_seconds":3.801,"snapshot":{"router_decisions":[{"need_memory":false,"page_ids":[],"record_ids":[],"page_scores":[],"record_scores":[],"hop_count":0,"hop_trace":[],"confidence":0.000001446954911443754,"stop_reason":"router_abstained","top_score":0,"score_margin":0,"evidence_score":0}],"no_evidence":false,"prefix_tokens":0,"prefix_used":false,"read_seconds":1.637925200000609,"auto_memory_probability":null,"auto_forget_probability":[0.011167768388986588],"context_compaction":{"compacted":false,"archived_records":0,"archived_tokens":0,"original_tokens":[1955],"retained_tokens":[1955],"hot_limit":32768,"archive_requested":true},"coverage":{"configured":true,"loaded":true,"bound_to_bank":true,"vocabulary_fraction":0.9,"head":{"path":"H:\\Memory\\V2_dpskw\\qwen3_5_4b_natural_memory_v2_1\\memory_attribute_head.pt","attributes":24,"vocabulary":["主管姓名","入职年份","出生城市","办公城市","午餐偏好","咖啡口味","团队名称","客户名称","宿舍楼号","工位楼层","常住城市","常用编辑器","手机尾号","档案标识","紧急联系人姓氏","设备型号","课程名称","起床时间","运动习惯","通勤方式","邮箱域名","阅读工具","项目代号","默认语言"],"coverage_stats":{"applicable":0,"bypassed":135}}},"memory":{"pages":0,"max_pages":32768,"page_capacity":32,"capacity_records":1048576,"open_pages":0,"records":0,"active_records":0,"superseded_records":0,"retracted_records":0,"quarantined_records":0,"hot_pages":0,"warm_pages":0,"cold_pages":0,"active_conflict_keys":0,"coarse_index_buckets":0,"last_coarse_candidates":0,"storage_mode":"embedded","resident_records":0,"resident_pages":0,"gpu_cache_records":0,"gpu_cache_tokens":0,"gpu_cache_bytes":0,"gpu_cache_hits":0,"gpu_cache_misses":0,"gpu_cache_fallbacks":0,"gpu_cache_alloc_failures":0,"gpu_cache_device":"cuda:0","gpu_cache_limit_records":256,"gpu_cache_limit_tokens":131072,"gpu_cache_reserve_mb":2048,"gpu_cache_adaptive":true,"gpu_cache_last_free_mb":null,"kv_budget":{"max_tokens":32768,"trigger_tokens":29491},"enabled":true,"router_ready":true}}},"memoryAfter":{"pages":0,"max_pages":32768,"page_capacity":32,"capacity_records":1048576,"open_pages":0,"records":0,"active_records":0,"superseded_records":0,"retracted_records":0,"quarantined_records":0,"hot_pages":0,"warm_pages":0,"cold_pages":0,"active_conflict_keys":0,"coarse_index_buckets":0,"last_coarse_candidates":0,"storage_mode":"embedded","resident_records":0,"resident_pages":0,"gpu_cache_records":0,"gpu_cache_tokens":0,"gpu_cache_bytes":0,"gpu_cache_hits":0,"gpu_cache_misses":0,"gpu_cache_fallbacks":0,"gpu_cache_alloc_failures":0,"gpu_cache_device":"cuda:0","gpu_cache_limit_records":256,"gpu_cache_limit_tokens":131072,"gpu_cache_reserve_mb":2048,"gpu_cache_adaptive":true,"gpu_cache_last_free_mb":null,"kv_budget":{"max_tokens":32768,"trigger_tokens":29491},"enabled":true,"router_ready":true}}
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1 @@
|
||||
我的工位在 3 楼东侧。
|
||||
@@ -0,0 +1,137 @@
|
||||
{
|
||||
"baseUrl": "http://127.0.0.1:8766/v1",
|
||||
"outDir": "H:\\Memory\\agent_lab\\agent\\runs\\suite5",
|
||||
"arms": [
|
||||
"on",
|
||||
"off"
|
||||
],
|
||||
"scenarios": [
|
||||
{
|
||||
"id": "conflict_update",
|
||||
"description": "同一属性先给旧值再给新值；跨会话询问时必须返回新值，且旧值不得作为 active 事实被读取。",
|
||||
"arms": {
|
||||
"on": {
|
||||
"activeRecords": 1,
|
||||
"assertionsPassed": 3,
|
||||
"assertionsTotal": 3,
|
||||
"passRate": 1,
|
||||
"failures": [],
|
||||
"meanSecondsPerTurn": 7.84
|
||||
},
|
||||
"off": {
|
||||
"activeRecords": 0,
|
||||
"assertionsPassed": 1,
|
||||
"assertionsTotal": 3,
|
||||
"passRate": 0.3333333333333333,
|
||||
"failures": [
|
||||
"answer_contains: missing one of [\"葡萄牙语\",\"葡萄牙\"]",
|
||||
"active_record_contains: absent from active memory"
|
||||
],
|
||||
"meanSecondsPerTurn": 6.16
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "cross_session_recall",
|
||||
"description": "写入事实后，在全新会话（空上下文、无历史文本）中询问该事实。这正是 in-model memory 存在的理由。",
|
||||
"arms": {
|
||||
"on": {
|
||||
"activeRecords": 2,
|
||||
"assertionsPassed": 3,
|
||||
"assertionsTotal": 3,
|
||||
"passRate": 1,
|
||||
"failures": [],
|
||||
"meanSecondsPerTurn": 8.7
|
||||
},
|
||||
"off": {
|
||||
"activeRecords": 0,
|
||||
"assertionsPassed": 0,
|
||||
"assertionsTotal": 3,
|
||||
"passRate": 0,
|
||||
"failures": [
|
||||
"answer_contains: missing one of [\"蓝鲸-47\",\"蓝鲸 47\",\"蓝鲸47\"]",
|
||||
"answer_contains: missing one of [\"ap-southeast-3\",\"apse-3\"]",
|
||||
"active_record_contains: absent from active memory"
|
||||
],
|
||||
"meanSecondsPerTurn": 5.84
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "forget_retraction",
|
||||
"description": "写入事实后明确要求遗忘；跨会话再问时必须不再使用该事实（撤回/覆盖语义），且不得把已忘内容当现状。",
|
||||
"arms": {
|
||||
"on": {
|
||||
"activeRecords": 0,
|
||||
"assertionsPassed": 1,
|
||||
"assertionsTotal": 2,
|
||||
"passRate": 0.5,
|
||||
"failures": [
|
||||
"answer_contains: missing one of [\"没有\",\"不知道\",\"已\",\"无法确认\",\"未保留\"]"
|
||||
],
|
||||
"meanSecondsPerTurn": 21.98
|
||||
},
|
||||
"off": {
|
||||
"activeRecords": 0,
|
||||
"assertionsPassed": 1,
|
||||
"assertionsTotal": 2,
|
||||
"passRate": 0.5,
|
||||
"failures": [
|
||||
"answer_contains: missing one of [\"没有\",\"不知道\",\"已\",\"无法确认\",\"未保留\"]"
|
||||
],
|
||||
"meanSecondsPerTurn": 14.13
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "multi_hop_chain",
|
||||
"description": "两条事实构成推理链（项目→负责人→工号）。跨会话问最终属性时，模型必须串起两跳而不是只取一跳。",
|
||||
"arms": {
|
||||
"on": {
|
||||
"activeRecords": 4,
|
||||
"assertionsPassed": 1,
|
||||
"assertionsTotal": 2,
|
||||
"passRate": 0.5,
|
||||
"failures": [
|
||||
"answer_contains: missing one of [\"H-9F3K2\",\"H 9F3K2\",\"9F3K2\"]"
|
||||
],
|
||||
"meanSecondsPerTurn": 24.39
|
||||
},
|
||||
"off": {
|
||||
"activeRecords": 0,
|
||||
"assertionsPassed": 1,
|
||||
"assertionsTotal": 2,
|
||||
"passRate": 0.5,
|
||||
"failures": [
|
||||
"answer_contains: missing one of [\"H-9F3K2\",\"H 9F3K2\",\"9F3K2\"]"
|
||||
],
|
||||
"meanSecondsPerTurn": 22.15
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "unknown_abstention",
|
||||
"description": "只写入一条无关事实，然后询问从未提及的属性。模型必须明确表示不知道，而不是编造或把相似记忆硬当答案。",
|
||||
"arms": {
|
||||
"on": {
|
||||
"activeRecords": 2,
|
||||
"assertionsPassed": 3,
|
||||
"assertionsTotal": 3,
|
||||
"passRate": 1,
|
||||
"failures": [],
|
||||
"meanSecondsPerTurn": 5.61
|
||||
},
|
||||
"off": {
|
||||
"activeRecords": 0,
|
||||
"assertionsPassed": 2,
|
||||
"assertionsTotal": 3,
|
||||
"passRate": 0.6666666666666666,
|
||||
"failures": [
|
||||
"answer_contains: missing one of [\"不知道\",\"没有记录\",\"无法确认\",\"未找到\",\"不清楚\"]"
|
||||
],
|
||||
"meanSecondsPerTurn": 17.69
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user