Initial commit: Natural Memory Agent Lab:模型服务 + TypeScript agent + 记忆能力场景评测(冲突更新、跨会话回忆、遗忘撤回、多跳、未知拒答)
This commit is contained in:
@@ -0,0 +1,31 @@
|
||||
{
|
||||
"id": "cross_session_recall",
|
||||
"description": "写入事实后,在全新会话(空上下文、无历史文本)中询问该事实。这正是 in-model memory 存在的理由。",
|
||||
"memory_write_turns": [0],
|
||||
"sessions": [
|
||||
{
|
||||
"id": "s1",
|
||||
"turns": [
|
||||
"记一下:我的项目代号是 蓝鲸-47,部署区域是 ap-southeast-3。",
|
||||
"谢谢。"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "s2",
|
||||
"turns": [
|
||||
"我的项目代号是什么?只回答代号本身。"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "s3",
|
||||
"turns": [
|
||||
"部署区域呢?只回答区域本身。"
|
||||
]
|
||||
}
|
||||
],
|
||||
"assertions": [
|
||||
{ "type": "answer_contains", "session": "s2", "turn": 0, "any": ["蓝鲸-47", "蓝鲸 47", "蓝鲸47"] },
|
||||
{ "type": "answer_contains", "session": "s3", "turn": 0, "any": ["ap-southeast-3", "apse-3"] },
|
||||
{ "type": "active_record_contains", "any": ["蓝鲸-47"] }
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user