Initial commit: Natural Memory Agent Lab:模型服务 + TypeScript agent + 记忆能力场景评测(冲突更新、跨会话回忆、遗忘撤回、多跳、未知拒答)
This commit is contained in:
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"id": "conflict_update",
|
||||
"description": "同一属性先给旧值再给新值;跨会话询问时必须返回新值,且旧值不得作为 active 事实被读取。",
|
||||
"sessions": [
|
||||
{
|
||||
"id": "s1",
|
||||
"turns": [
|
||||
"记一下:我的默认语言是 日语。",
|
||||
"更正一下:我的默认语言改成 葡萄牙语,日语作废。"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "s2",
|
||||
"turns": [
|
||||
"我的默认语言是什么?只回答语言名。"
|
||||
]
|
||||
}
|
||||
],
|
||||
"assertions": [
|
||||
{ "type": "answer_contains", "session": "s2", "turn": 0, "any": ["葡萄牙语", "葡萄牙"] },
|
||||
{ "type": "answer_not_contains", "session": "s2", "turn": 0, "any": ["日语"] },
|
||||
{ "type": "active_record_contains", "any": ["葡萄牙语"] }
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user