feat: release v0.3.3 — Hermes adapter, context offload, core refactor

This commit is contained in:
chrishuan
2026-05-13 01:58:18 +08:00
parent a74b0b3e43
commit db8f3e516a
100 changed files with 29832 additions and 454 deletions
+342
View File
@@ -0,0 +1,342 @@
# memory-tencentdb-ctl.sh — memory_tencentdb (TDAI) 运维脚本
> 配合 [`install_hermes_memory_tencentdb.sh`](./install_hermes_memory_tencentdb.sh) 使用。
> 先跑安装脚本部署好插件 + Node 依赖,之后日常的启停 / 配置全部通过 `memory-tencentdb-ctl.sh` 完成。
## 1. 运行模式
脚本有两种模式,**默认是独立模式**,完全不触碰 `~/.hermes`
| 模式 | 激活方式 | 做什么 | 不做什么 |
|---|---|---|---|
| `standalone`(默认) | 无需任何参数 | 启停 Gateway;写 `$TDAI_DATA_DIR/tdai-gateway.json`;日志落 `$TDAI_DATA_DIR/logs/` | 不写 `$HERMES_HOME/env.d/`,不改 `$HERMES_HOME/config.yaml`,不读 hermes 相关 env |
| `hermes` | 命令行追加 `--hermes`,或环境 `MEMORY_TENCENTDB_MODE=hermes` | standalone 的全部 + `config llm` 同步写 `$HERMES_HOME/env.d/memory-tencentdb-llm.sh`;日志落 `$HERMES_HOME/logs/memory_tencentdb/`;开放 `enable-hermes-memory` 子命令 | — |
> **为什么 hermes 模式要多写 env 文件?**
> 因为 hermes 进程会托管式地把 Gateway 以子进程拉起来(supervisor 用 `os.environ.copy()` 传环境),此时 Gateway 读不到 `tdai-gateway.json` 所在的 shell 环境,必须通过 `$HERMES_HOME/env.d/*.sh` 让 hermes 自身 `source` 才能把凭据传进去。独立模式下 Gateway 自己读 JSON,无此需要。
## 2. 路径
> **路径变量约定**:本节及后续示例统一用 `$HERMES_HOME` 指代 hermes 的家目录,**默认 `~/.hermes`**,但你可以通过环境变量覆盖(例如 `export HERMES_HOME=/srv/hermes`),脚本和 hermes 自身都遵守这个变量。
>
> 自 0.4.x 起,所有 tdai 相关数据/代码默认收纳到统一根目录 `$MEMORY_TENCENTDB_ROOT`(默认 `~/.memory-tencentdb`)之下:
>
> - `$TDAI_INSTALL_DIR` 默认 `$MEMORY_TENCENTDB_ROOT/tdai-memory-openclaw-plugin`(即 `~/.memory-tencentdb/tdai-memory-openclaw-plugin`
> - `$TDAI_DATA_DIR` 默认 `$MEMORY_TENCENTDB_ROOT/memory-tdai`(即 `~/.memory-tencentdb/memory-tdai`
>
> 下文出现这些变量时**先 `export` 一份覆盖**,再跑命令即可全局生效。
> 旧版本使用 `~/tdai-memory-openclaw-plugin` 与 `~/memory-tdai``install_hermes_memory_tencentdb.sh` 在升级时会自动迁移这两个旧目录到新位置。
| 路径 | standalone | hermes | 作用 |
|---|---|---|---|
| `$TDAI_INSTALL_DIR` | ✅ | ✅ | 插件源码 + `node_modules` + `src/gateway/server.ts` |
| `$TDAI_DATA_DIR/tdai-gateway.json` | ✅ | ✅ | Gateway 主配置:`llm` / `memory.embedding` / `memory.tcvdb` / `memory.storeBackend`,权限 `0600` |
| `$TDAI_DATA_DIR/logs/` | ✅ 日志 | — | `gateway.stdout.log` / `gateway.stderr.log` / `gateway.pid` |
| `$HERMES_HOME/logs/memory_tencentdb/` | — | ✅ 日志 | 同上,换目录 |
| `$HERMES_HOME/env.d/memory-tencentdb-llm.sh` | — | ✅ | hermes 启动前 `source`,给 supervisor 托管的 Gateway 子进程注入 LLM 凭据 |
| `$HERMES_HOME/config.yaml` | — | ✅ | `enable-hermes-memory` 修改其 `memory.provider` |
| Gateway 监听 | `127.0.0.1:8420` | `127.0.0.1:8420` | 可被 `MEMORY_TENCENTDB_GATEWAY_HOST/PORT` 覆盖 |
所有路径都能用同名环境变量覆盖(再次列出便于对照):`MEMORY_TENCENTDB_ROOT`(默认 `~/.memory-tencentdb`)、`TDAI_INSTALL_DIR`(默认 `$MEMORY_TENCENTDB_ROOT/tdai-memory-openclaw-plugin`)、`TDAI_DATA_DIR`(默认 `$MEMORY_TENCENTDB_ROOT/memory-tdai`)、`HERMES_HOME`(默认 `~/.hermes`)、`MEMORY_TENCENTDB_LOG_DIR``MEMORY_TENCENTDB_GATEWAY_HOST/PORT`
依赖:`bash``python3``node >= 22``npx``lsof``ss`
## 3. 安装 & 调用
脚本随 npm 包发布到 `node_modules/.../scripts/` 下,但**没有**注册为 `bin` 命令。要用全局命令名调用,必须自己做一次软链。
### 3.1 从 npm 包里直接运行(无需任何配置)
```bash
npm install @tencentdb-agent-memory/memory-tencentdb
# 项目内安装,路径可由 npm root 动态算出
"$(npm root)/@tencentdb-agent-memory/memory-tencentdb/scripts/memory-tencentdb-ctl.sh" --help
# 全局安装则用 npm root -g
"$(npm root -g)/@tencentdb-agent-memory/memory-tencentdb/scripts/memory-tencentdb-ctl.sh" --help
```
`npm root` / `npm root -g` 会在所有包管理器(npm / pnpm / yarn)和不同的 prefix 配置下返回正确目录,避免硬编码 `node_modules/` 路径。适合一次性、临时使用的场景。
### 3.2 软链到 PATH(推荐给运维 / 长期使用)
不论脚本来源(git clone 出来的仓库 / `npm install` 装的包 / 自定义部署目录),先把脚本路径**算出来存到一个变量里**,再统一做软链。这样无需关心你把仓库放在 `~/code/``/opt/`、还是别的什么地方。
```bash
# 第一步:定位 memory-tencentdb-ctl.sh 的真实路径(任选一种来源)
# (a) 从 git 仓库(在仓库根目录或任意子目录里执行)
SCRIPT="$(git -C "$(git rev-parse --show-toplevel)" ls-files | \
grep -E 'scripts/memory-tencentdb-ctl\.sh$' | head -1)"
SCRIPT="$(git rev-parse --show-toplevel)/$SCRIPT"
# (b) 从已 npm 全局安装的包
SCRIPT="$(npm root -g)/@tencentdb-agent-memory/memory-tencentdb/scripts/memory-tencentdb-ctl.sh"
# (c) 从项目本地的 node_modules
SCRIPT="$(npm root)/@tencentdb-agent-memory/memory-tencentdb/scripts/memory-tencentdb-ctl.sh"
# (d) 完全手写绝对路径(如部署到非标准位置)
SCRIPT="/opt/tdai/scripts/memory-tencentdb-ctl.sh"
# 第二步:验证路径正确,然后软链
test -f "$SCRIPT" && echo "ok: $SCRIPT" || { echo "not found"; exit 1; }
chmod +x "$SCRIPT"
sudo ln -sf "$SCRIPT" /usr/local/bin/memory-tencentdb-ctl
# 同样的办法把 install_hermes_memory_tencentdb.sh 链接成 install-memory-tencentdb(可选)
INSTALL_SCRIPT="$(dirname "$SCRIPT")/install_hermes_memory_tencentdb.sh"
test -f "$INSTALL_SCRIPT" && {
chmod +x "$INSTALL_SCRIPT"
sudo ln -sf "$INSTALL_SCRIPT" /usr/local/bin/install-memory-tencentdb
}
```
之后直接 `memory-tencentdb-ctl …` / `install-memory-tencentdb …`
> **为什么不直接用 `npm bin` 注册?** 这两个脚本是运维工具而不是包的核心 API,主仓库希望用户**显式**完成 PATH 注册(避免无意中污染全局命令空间,并避免 npm 卸载时静默移除运维入口)。
## 4. 生命周期管理(两种模式通用)
```bash
memory-tencentdb-ctl start # 若 :8420 已占用会直接返回;否则后台 spawn,等待 /health 通过
memory-tencentdb-ctl stop # 先 SIGTERM5s 内未退则 SIGKILL
memory-tencentdb-ctl restart
memory-tencentdb-ctl status # 打印模式、端口、data/log 路径、进程状态
memory-tencentdb-ctl health # GET /health,纯 python3 实现,不要求 curl
memory-tencentdb-ctl logs # tail -f stdout + stderr
memory-tencentdb-ctl logs err 500 # 只看 stderr 最近 500 行
```
启动命令解析顺序:
1. 环境变量 `MEMORY_TENCENTDB_GATEWAY_CMD``install_hermes_memory_tencentdb.sh` 写入 `/etc/profile.d/memory-tencentdb-env.sh` 的那条)。
2. 回退到 `sh -c 'cd $TDAI_INSTALL_DIR && exec npx tsx src/gateway/server.ts'`
启动时会自动 `source` 的环境文件:
- 两种模式:`/etc/profile.d/memory-tencentdb-env.sh`
- 仅 hermes 模式:`/etc/profile.d/hermes-env.sh` 以及 `$HERMES_HOME/env.d/*.sh`
## 5. 配置 LLM / Embedding / VDB
三类凭据统一落到 `$TDAI_DATA_DIR/tdai-gateway.json``0600`,原子写)。**`config llm``--hermes` 模式下会额外写一份 env 文件**Embedding / VDB 从不写 env。
### 5.1 LLM
```bash
# standalone 模式:只写 tdai-gateway.json
memory-tencentdb-ctl config llm \
--api-key "sk-xxxxxxxxxxxx" \
--base-url "https://api.openai.com/v1" \
--model "gpt-4o" \
--restart
# hermes 模式:tdai-gateway.json + $HERMES_HOME/env.d/memory-tencentdb-llm.sh
memory-tencentdb-ctl --hermes config llm \
--api-key "sk-xxxxxxxxxxxx" \
--base-url "https://api.openai.com/v1" \
--model "gpt-4o" \
--restart
```
- JSON 写入点:`$.llm.{baseUrl, apiKey, model}`
- env 文件写入(仅 `--hermes`):`TDAI_LLM_*``MEMORY_TENCENTDB_LLM_*` 别名(Python provider 的 `get_config_schema()` 会读后者)。
### 5.2 Embedding
默认关闭(`provider=none`)。启用远端 OpenAI 兼容服务:
```bash
memory-tencentdb-ctl config embedding \
--provider openai \
--api-key "sk-xxxx" \
--base-url "https://api.openai.com/v1" \
--model "text-embedding-3-small" \
--dimensions 1536 \
--restart
# 关闭 embedding(退化为 BM25/关键词召回)
memory-tencentdb-ctl config embedding --provider none --restart
```
- JSON 写入点:`$.memory.embedding.{provider, baseUrl, apiKey, model, dimensions, enabled, proxyUrl?}`
- `qclaw` provider 额外要求 `--proxy-url`
- 校验规则与 `src/config.ts``parseConfig()` 对齐:`dimensions` 为正整数,非 `none` 必须带 `apiKey/baseUrl/model/dimensions`;缺项直接报错不写半残 JSON。
### 5.3 VectorDBTencent Cloud VDB / tcvdb
```bash
memory-tencentdb-ctl config vdb \
--url "http://xxx-vdb.tencentclb.com:8100" \
--username root \
--api-key "YOUR-VDB-API-KEY" \
--database "openclaw_memory" \
--alias "primary" \
--embedding-model "bge-large-zh" \
--ca-pem "/etc/ssl/vdb-ca.pem" \
--restart
```
- JSON 写入点:`$.memory.tcvdb.{url, username, apiKey, database, alias?, caPemPath?, embeddingModel?}`
- 默认同时把 `$.memory.storeBackend` 切到 `"tcvdb"`;只想预埋配置先不切,加 `--no-set-backend`
- `--ca-pem` 只写路径不复制文件;脚本会校验可读性。
### 5.4 退回本地 SQLite(关闭 VDB 后端)
```bash
# 默认:保留 memory.tcvdb 凭据(方便随时再切回去),仅把 storeBackend 改回 sqlite
memory-tencentdb-ctl config vdb-off --restart
# 同时把腾讯云 VDB 的 url / apiKey / database 等凭据从 JSON 中清掉
memory-tencentdb-ctl config vdb-off --purge-creds --restart
```
- JSON 写入点:把 `$.memory.storeBackend` 设为 `"sqlite"``--purge-creds` 时额外删除整段 `$.memory.tcvdb`
- `$.llm` / `$.memory.embedding` 等其它顶级段**完全保留**,hermes 侧 `memory.provider` **不动**(仍是 `memory_tencentdb`,只是它内部存储退回 sqlite)。
- 配置文件不存在时给出 `warn` 并写入仅含 `{"memory":{"storeBackend":"sqlite"}}` 的最小配置。
-`config vdb` 完全镜像:可与 `--dry-run` / `--restart` 组合使用。
### 5.5 查看当前配置
```bash
memory-tencentdb-ctl config show
```
- 打印 `tdai-gateway.json``apiKey`/`password`/`token` 字段自动脱敏为 `<redacted:NN chars>`
- hermes 模式下额外打印 `$HERMES_HOME/env.d/memory-tencentdb-*.sh`(API key 也会脱敏),可直接贴工单。
## 6. 打通 hermes(仅 `--hermes` 模式)
```bash
memory-tencentdb-ctl --hermes enable-hermes-memory
```
幂等:把 `$HERMES_HOME/config.yaml``memory:` 段的 `provider:` 改成 `memory_tencentdb`(不存在则新增整段)。改完后重启 hermes:
```bash
source "$HERMES_HOME/env.d/memory-tencentdb-llm.sh"
pkill -f hermes-agent || true
hermes
```
> **关于写入策略**:脚本采用"格式保真"双路径,**永不重写整个 YAML**:
>
> 1. **首选**:检测到 [`ruamel.yaml`](https://yaml.readthedocs.io/) 时走 round-trip,完整保留注释、键序、引号、缩进风格(推荐 `pip install --user ruamel.yaml` 享受最佳保真度,**非必装**);
> 2. **降级**:未安装 ruamel 时走最小化原位行编辑——只重写 `provider:` 那一行,缩进直接从同段已有兄弟键的前缀**逐字符拷贝**(零猜测、零格式破坏);
> 3. 若 `memory:` 段不存在,则在文件末尾追加最小段,缩进从文档其它顶级段的子键拓印。
>
> 实测对真实 `~/.hermes/config.yaml` 做 byte-for-byte diff,除 `provider` 值本身外其余字节完全一致。
非 hermes 模式下调用该命令会直接报错退出。
> 想反过来"保留 hermes provider 不变,仅让 TDAI 内部存储退回 sqlite",用 §5.4 的 `config vdb-off` 即可(不需要也**不要**改 hermes 的 `memory.provider`)。
## 7. 典型使用流程
### 场景 AGateway 独立部署(不使用 hermes
```bash
# 1) 安装
# INSTALL_SCRIPT 的取值方式见上文 3.2 节(git rev-parse / npm root / 手填均可)
# 例如从 git 仓库根:INSTALL_SCRIPT="$(git rev-parse --show-toplevel)/scripts/install_hermes_memory_tencentdb.sh"
# 从 npm 全局: INSTALL_SCRIPT="$(npm root -g)/@tencentdb-agent-memory/memory-tencentdb/scripts/install_hermes_memory_tencentdb.sh"
bash "$INSTALL_SCRIPT"
# 2) 只配 Gateway 所需凭据
memory-tencentdb-ctl config llm --api-key "sk-..." --base-url "https://api.openai.com/v1" --model gpt-4o
memory-tencentdb-ctl config embedding --provider openai --api-key "sk-..." --base-url "https://api.openai.com/v1" \
--model text-embedding-3-small --dimensions 1536
memory-tencentdb-ctl config vdb --url "http://xxx:8100" --api-key "..." --database openclaw_memory
# 3) 启动 + 自检
memory-tencentdb-ctl start
memory-tencentdb-ctl status
memory-tencentdb-ctl health # 预期: {"status":"ok",...}
```
### 场景 B:集成 hermes
```bash
# 1) 安装(与场景 A 相同;INSTALL_SCRIPT 由上文 3.2 节算出)
bash "$INSTALL_SCRIPT"
# 2) 全程加 --hermes(或 export MEMORY_TENCENTDB_MODE=hermes 一次)
memory-tencentdb-ctl --hermes config llm --api-key "sk-..." --base-url "https://api.openai.com/v1" --model gpt-4o
memory-tencentdb-ctl --hermes config embedding --provider openai --api-key "sk-..." \
--base-url "https://api.openai.com/v1" \
--model text-embedding-3-small --dimensions 1536
memory-tencentdb-ctl --hermes config vdb --url "http://xxx:8100" --api-key "..." --database openclaw_memory
# 3) 启动 Gateway(通常由 hermes supervisor 托管,这里是手动兜底)
memory-tencentdb-ctl --hermes start
memory-tencentdb-ctl --hermes status
# 4) 在 hermes config 里启用 provider,并重启 hermes
memory-tencentdb-ctl --hermes enable-hermes-memory
source "$HERMES_HOME/env.d/memory-tencentdb-llm.sh"
pkill -f hermes-agent ; hermes
```
如果嫌 `--hermes` 每次都要带,可以:
```bash
export MEMORY_TENCENTDB_MODE=hermes
```
之后所有调用自动切到 hermes 模式,命令行无需再加 `--hermes`
### 场景 C:临时把 TDAI 的存储退回 sqlite(保留 hermes 集成)
适用于 VDB 不可达 / 排障 / 离线开发等场景:希望 hermes 端 `memory.provider` 仍是 `memory_tencentdb`,但让 Gateway 改用本地 SQLite 落盘。
```bash
# (A) 默认:保留 memory.tcvdb 凭据,仅把 storeBackend 切回 sqlite
memory-tencentdb-ctl config vdb-off --restart
# (B) 排障结束想切回 vdb:当前需要重新跑一次 config vdb(必填项需重新提供,
# 即使 JSON 里凭据还在;脚本是基于必填校验的"重新声明"语义,不是 toggle
memory-tencentdb-ctl config vdb \
--url "http://xxx-vdb.tencentclb.com:8100" \
--api-key "<你的 KEY>" \
--database "openclaw_memory" \
--restart
# (C) 彻底放弃 vdb:清掉凭据
memory-tencentdb-ctl config vdb-off --purge-creds --restart
```
> 之所以 (B) 不提供"零参数 vdb-on",是因为原 `config vdb` 子命令把 `--url/--api-key/--database` 设为强校验,避免用户拼装出半残配置;如果你希望把"已存的凭据重新激活"也做成单命令,告诉维护者补一个 `config vdb-on` 即可(实现方式与 `vdb-off` 完全镜像)。
> **不要**为了"切回 sqlite"去改 `~/.hermes/config.yaml` 的 `memory.provider`hermes 看到的依然是 `memory_tencentdb` provider,存储后端切换是 Gateway 内部的事,对 hermes 完全透明。
## 8. 全局选项 & 调试技巧
- 所有写操作支持 `--dry-run`(放在命令最前),会打印将要写入的内容但不落盘:
```bash
memory-tencentdb-ctl --dry-run config llm --api-key k --base-url https://x --model m
```
- 敏感文件权限一律 `0600``env.d/memory-tencentdb-llm.sh` 含明文 API key**不要** commit。
- 启动失败:`memory-tencentdb-ctl logs err 200` 查看 stderr;手动前台跑一遍更容易看到报错:
```bash
cd "$TDAI_INSTALL_DIR" && npx tsx src/gateway/server.ts
```
- 端口冲突:`MEMORY_TENCENTDB_GATEWAY_PORT=18420 memory-tencentdb-ctl restart`。
- 验证 hermes 有没有吃到新 envhermes 模式):
```bash
tr '\0' '\n' < /proc/$(pgrep -n hermes-agent)/environ | grep -E 'TDAI_|MEMORY_TENCENTDB_'
```
## 9. 退出码
| 码 | 含义 |
|---|---|
| 0 | 成功 |
| 1 | 参数错误 / 业务校验失败(如 `--base-url` 非 http(s);在 standalone 下调 hermes 专属命令) |
| 2 | 写盘失败(磁盘满、权限不足等) |
| 127 | 依赖缺失(`python3` / `node` / `npx` |
---
要封装成 systemd unit,基于 `memory-tencentdb-ctl start` / `memory-tencentdb-ctl stop` 写 `Type=forking` 的 service 即可(Gateway 是无状态 HTTP sidecar,不依赖 systemd readiness 协议)。
+332
View File
@@ -0,0 +1,332 @@
#!/bin/bash
#
# install_memory_tencentdb.sh
#
# 在 install_hermes_ubuntu.sh 之后执行,用于:
# 1. 通过 npm 下载 @tencentdb-agent-memory/memory-tencentdb@latest 到
# $MEMORY_TENCENTDB_ROOT/tdai-memory-openclaw-plugin(默认 ~/.memory-tencentdb/tdai-memory-openclaw-plugin
# 2. 安装 Gateway 的 Node.js 依赖(npm install
# 3. 配置 hermes config.yaml 使用 memory_tencentdb 记忆提供者
# 4. 设置 Gateway 自动启动环境变量
#
# 路径约定(全部位于 ~/.memory-tencentdb/ 之下,可通过环境变量覆盖):
# $MEMORY_TENCENTDB_ROOT 默认 ~/.memory-tencentdb
# $TDAI_INSTALL_DIR 默认 $MEMORY_TENCENTDB_ROOT/tdai-memory-openclaw-plugin
# $TDAI_DATA_DIR 默认 $MEMORY_TENCENTDB_ROOT/memory-tdai
#
# 旧版本(<= 0.3.x)使用 ~/tdai-memory-openclaw-plugin 与 ~/memory-tdai
# 本脚本会在执行前自动迁移这两个旧目录到新位置(见 Step 0)。
#
# 使用方式:
# 以目标用户身份执行(推荐):
# su - <username> -c "bash ~/install_memory_tencentdb.sh"
# # 或直接以该用户登录后执行
# bash ~/install_memory_tencentdb.sh
#
# 以 root 身份执行(镜像构建场景):
# bash ~/install_memory_tencentdb.sh
# # root 会自动 su 切换到目标用户执行,完成后修复权限
#
# 前置条件:
# - install_hermes_ubuntu.sh 已执行完成(hermes-agent 已安装)
# - Node.js >= 22 已安装
set -e
# 动态获取当前执行用户及 HOME 目录
USERNAME=$(whoami)
USER_HOME=$(eval echo ~$USERNAME)
# npm 包名
NPM_PACKAGE="@tencentdb-agent-memory/memory-tencentdb@latest"
# Hermes 路径
HERMES_HOME="$USER_HOME/.hermes"
HERMES_AGENT_DIR="$HERMES_HOME/hermes-agent"
HERMES_CONFIG="$HERMES_HOME/config.yaml"
# memory-tencentdb 统一根目录(所有 tdai 相关数据/代码都收纳在此)
# 可通过环境变量 MEMORY_TENCENTDB_ROOT 覆盖
MEMORY_TENCENTDB_ROOT="${MEMORY_TENCENTDB_ROOT:-$USER_HOME/.memory-tencentdb}"
# tdai 解压目标目录(位于统一根目录下)
# 可通过环境变量 TDAI_INSTALL_DIR 覆盖
TDAI_INSTALL_DIR="${TDAI_INSTALL_DIR:-$MEMORY_TENCENTDB_ROOT/tdai-memory-openclaw-plugin}"
# tdai 数据目录(Gateway baseDir,位于统一根目录下)
# 可通过环境变量 TDAI_DATA_DIR 覆盖
TDAI_DATA_DIR="${TDAI_DATA_DIR:-$MEMORY_TENCENTDB_ROOT/memory-tdai}"
# 旧路径(仅用于自动迁移)
LEGACY_INSTALL_DIR="$USER_HOME/tdai-memory-openclaw-plugin"
LEGACY_DATA_DIR="$USER_HOME/memory-tdai"
# ==================== root → 自动切换到目标用户 ====================
# 与 install_hermes_ubuntu.sh 保持一致:如果以 root 执行,自动 su 切到
# 目标用户运行实际安装逻辑。也可以直接以目标用户身份执行,跳过此段。
if [ "$(id -u)" -eq 0 ]; then
echo "[memory-tencentdb] Running as root, switching to $USERNAME for installation..."
# 验证前置条件
if [ ! -d "$HERMES_AGENT_DIR" ]; then
echo "[ERROR] Hermes agent not found at $HERMES_AGENT_DIR"
echo "[ERROR] Please run install_hermes_ubuntu.sh first."
exit 1
fi
# 切换到目标用户执行
TEMP_SCRIPT=$(mktemp /tmp/memory-tencentdb-install-XXXXXX.sh)
cp "${BASH_SOURCE[0]}" "$TEMP_SCRIPT"
chmod 755 "$TEMP_SCRIPT"
su - $USERNAME -c "bash $TEMP_SCRIPT" </dev/null
# 修复权限
echo "[memory-tencentdb] Fixing permissions..."
chown -R $USERNAME:$USERNAME "$USER_HOME"
rm -f "$TEMP_SCRIPT"
echo "[memory-tencentdb] Installation completed successfully"
exit 0
fi
# ==================== 用户阶段(核心安装逻辑) ====================
echo "[memory-tencentdb] Installing memory-tencentdb plugin (user: $(whoami))..."
# 验证前置条件
if [ ! -d "$HERMES_AGENT_DIR" ]; then
echo "[ERROR] Hermes agent not found at $HERMES_AGENT_DIR"
echo "[ERROR] Please run install_hermes_ubuntu.sh first."
exit 1
fi
# 加载 hermes 环境(PATH 中需要 node/npm
if [ -f /etc/profile.d/hermes-env.sh ]; then
source /etc/profile.d/hermes-env.sh
fi
# 确保统一根目录存在
mkdir -p "$MEMORY_TENCENTDB_ROOT"
# ---------- Step 0: 自动迁移旧路径(向后兼容) ----------
#
# 历史版本把 tdai 解压到 ~/tdai-memory-openclaw-plugin、数据放在 ~/memory-tdai。
# 现在统一收纳到 ~/.memory-tencentdb/ 之下,这里做一次性自动迁移。
# 已经在新位置时跳过;新旧都存在时打印警告并保留新位置不动。
migrate_legacy_dir() {
local legacy="$1"
local target="$2"
local label="$3"
if [ ! -e "$legacy" ]; then
return 0
fi
if [ -L "$legacy" ]; then
# 旧位置是 symlink,直接清掉
echo "[memory-tencentdb] Removing legacy symlink: $legacy"
rm -f "$legacy"
return 0
fi
if [ -e "$target" ]; then
echo "[memory-tencentdb] WARN: legacy $label dir exists at $legacy but new location $target also exists." >&2
echo "[memory-tencentdb] WARN: keeping new location; please review and remove $legacy manually if obsolete." >&2
return 0
fi
echo "[memory-tencentdb] Migrating legacy $label dir: $legacy -> $target"
mkdir -p "$(dirname "$target")"
mv "$legacy" "$target"
}
migrate_legacy_dir "$LEGACY_INSTALL_DIR" "$TDAI_INSTALL_DIR" "install"
migrate_legacy_dir "$LEGACY_DATA_DIR" "$TDAI_DATA_DIR" "data"
# ---------- Step 1: 通过 npm 下载包并提取到 $TDAI_INSTALL_DIR ----------
echo "[memory-tencentdb] Step 1: Downloading $NPM_PACKAGE via npm..."
# 清理旧安装
rm -rf "$TDAI_INSTALL_DIR"
# 使用临时目录通过 npm install 下载包
TEMP_DOWNLOAD=$(mktemp -d /tmp/memory-tencentdb-download-XXXXXX)
cd "$TEMP_DOWNLOAD"
npm init -y --silent > /dev/null 2>&1
npm install "$NPM_PACKAGE" --omit=dev 2>&1 | tail -5
# 包安装后位于 node_modules/@tencentdb-agent-memory/memory-tencentdb
PACK_DIR="$TEMP_DOWNLOAD/node_modules/@tencentdb-agent-memory/memory-tencentdb"
if [ ! -d "$PACK_DIR" ]; then
echo "[ERROR] Downloaded package directory not found at $PACK_DIR"
rm -rf "$TEMP_DOWNLOAD"
exit 1
fi
# 将包内容移动到目标安装目录
mkdir -p "$(dirname "$TDAI_INSTALL_DIR")"
cp -r "$PACK_DIR" "$TDAI_INSTALL_DIR"
echo "[memory-tencentdb] Package downloaded and extracted to $TDAI_INSTALL_DIR"
# ---------- Step 2: 安装 Gateway Node.js 依赖 ----------
echo "[memory-tencentdb] Step 2: Installing Gateway dependencies..."
cd "$TDAI_INSTALL_DIR"
echo "[memory-tencentdb] Running npm install (this may take a while)..."
npm install --omit=dev 2>&1 | tail -5
# 安装 tsx(Gateway 启动需要),优先本地安装
if ! npx tsx --version &>/dev/null; then
npm install tsx 2>&1 | tail -2
fi
echo "[memory-tencentdb] Gateway dependencies installed"
# ---------- Step 2.5: 将插件链接到 hermes 插件目录 ----------
echo "[memory-tencentdb] Step 2.5: Linking plugin into hermes plugins directory..."
HERMES_PLUGIN_DIR="$HERMES_AGENT_DIR/plugins/memory/memory_tencentdb"
PLUGIN_SRC_DIR="$TDAI_INSTALL_DIR/hermes-plugin/memory/memory_tencentdb"
# 移除旧链接/目录
rm -rf "$HERMES_PLUGIN_DIR"
# 创建 symlink 使 hermes 能发现插件
ln -sf "$PLUGIN_SRC_DIR" "$HERMES_PLUGIN_DIR"
echo "[memory-tencentdb] Plugin linked: $HERMES_PLUGIN_DIR -> $PLUGIN_SRC_DIR"
# ---------- Step 3: 提示用户手动开启 memory_tencentdb(不自动修改 config ----------
echo "[memory-tencentdb] Step 3: Checking hermes config..."
# 插件已链接到 hermes 插件目录,但默认不自动启用,仅提示
if [ -f "$HERMES_CONFIG" ]; then
if sed -n '/^memory:/,/^[a-zA-Z]/p' "$HERMES_CONFIG" | grep -q "provider: memory_tencentdb"; then
echo "[memory-tencentdb] memory.provider already set to memory_tencentdb"
else
echo "[memory-tencentdb] Plugin installed but NOT enabled by default."
echo "[memory-tencentdb] To enable tdai memory, add/edit in $HERMES_CONFIG:"
echo ""
echo " memory:"
echo " provider: memory_tencentdb"
echo ""
fi
else
echo "[memory-tencentdb] WARN: $HERMES_CONFIG not found, please run install_hermes_ubuntu.sh first"
fi
# ---------- Step 4: 配置 Gateway 环境变量 ----------
echo "[memory-tencentdb] Step 4: Setting up Gateway environment..."
# 构建 Gateway 启动命令
# 使用 sh -c 包裹,先 cd 到插件目录再启动 Gateway(ESM 解析需要)
GATEWAY_CMD="sh -c 'cd $TDAI_INSTALL_DIR && exec npx tsx src/gateway/server.ts'"
# ── 4a: /etc/profile.dSSH 交互式登录场景) ──
# 写入 /etc/profile.d 持久化环境变量,供 SSH 手动执行 `hermes` 时使用。
# 注意:LLM 相关变量(API key、model 等)需要用户后续手动配置
ENVFILE="/etc/profile.d/memory-tencentdb-env.sh"
cat << ENVEOF | sudo tee "$ENVFILE" > /dev/null
# memory-tencentdb Gateway 环境变量
export MEMORY_TENCENTDB_GATEWAY_CMD="$GATEWAY_CMD"
export MEMORY_TENCENTDB_GATEWAY_HOST="127.0.0.1"
export MEMORY_TENCENTDB_GATEWAY_PORT="8420"
# LLM 配置(按需修改)
# export MEMORY_TENCENTDB_LLM_API_KEY="sk-..."
# export MEMORY_TENCENTDB_LLM_BASE_URL="https://api.openai.com/v1"
# export MEMORY_TENCENTDB_LLM_MODEL="gpt-4o"
ENVEOF
echo "[memory-tencentdb] Environment variables written to $ENVFILE"
# ── 4b: ~/.hermes/.envsystemd service 场景) ──
# hermes-gateway 通过 systemd user service 启动时不会 source /etc/profile.d/*.sh
# 但 hermes 的 run.py 启动时会 load_dotenv("~/.hermes/.env")。
# 因此必须将关键变量同步写入 .env,否则 systemd 场景下 Gateway 无法自动启动。
HERMES_ENV="$HERMES_HOME/.env"
_append_or_update_env() {
local key="$1"
local value="$2"
local file="$3"
if [ ! -f "$file" ]; then
touch "$file"
fi
# 移除已有的同名变量行(含注释掉的和带引号的),再追加
sed -i "/^${key}=/d" "$file"
sed -i "/^# *${key}=/d" "$file"
# python-dotenv 要求含空格/引号/特殊字符的值用双引号包裹
echo "${key}=\"${value}\"" >> "$file"
}
_append_or_update_env "MEMORY_TENCENTDB_GATEWAY_CMD" "$GATEWAY_CMD" "$HERMES_ENV"
_append_or_update_env "MEMORY_TENCENTDB_GATEWAY_HOST" "127.0.0.1" "$HERMES_ENV"
_append_or_update_env "MEMORY_TENCENTDB_GATEWAY_PORT" "8420" "$HERMES_ENV"
echo "[memory-tencentdb] Gateway env vars also written to $HERMES_ENV (for systemd service)"
# ---------- 清理 ----------
rm -rf "$TEMP_DOWNLOAD"
# ---------- 验证安装 ----------
echo ""
echo "=========================================="
echo "[memory-tencentdb] Installation Summary"
echo "=========================================="
echo " Root dir: $MEMORY_TENCENTDB_ROOT"
echo " tdai source: $TDAI_INSTALL_DIR"
echo " tdai data dir: $TDAI_DATA_DIR"
echo " Hermes config: $HERMES_CONFIG"
echo " Env file: $ENVFILE"
echo ""
echo " Installed files in tdai dir:"
ls -la "$TDAI_INSTALL_DIR"/ 2>/dev/null | head -20 || echo " (none)"
echo ""
# 验证 hermes 插件文件存在(在解压目录中)
PLUGIN_SRC="$TDAI_INSTALL_DIR/hermes-plugin/memory/memory_tencentdb"
MISSING=0
for f in __init__.py plugin.yaml client.py supervisor.py; do
if [ ! -f "$PLUGIN_SRC/$f" ]; then
echo " [WARN] Missing: $PLUGIN_SRC/$f"
MISSING=1
fi
done
if [ "$MISSING" -eq 0 ]; then
echo " [OK] All hermes plugin files present"
fi
# 验证 Gateway 入口存在
if [ -f "$TDAI_INSTALL_DIR/src/gateway/server.ts" ]; then
echo " [OK] Gateway entry point found"
else
echo " [WARN] Gateway server.ts not found at $TDAI_INSTALL_DIR/src/gateway/server.ts"
fi
# 验证 node_modules 已安装
if [ -d "$TDAI_INSTALL_DIR/node_modules" ]; then
echo " [OK] Gateway node_modules installed"
else
echo " [WARN] Gateway node_modules not found"
fi
echo ""
echo "[memory-tencentdb] Done!"
echo ""
echo " NOTE: Before using the memory plugin, configure LLM credentials in ~/.hermes/.env:"
echo " MEMORY_TENCENTDB_LLM_API_KEY=your-api-key"
echo " MEMORY_TENCENTDB_LLM_BASE_URL=https://api.openai.com/v1"
echo " MEMORY_TENCENTDB_LLM_MODEL=gpt-4o"
echo ""
echo " (For systemd-managed hermes-gateway, ~/.hermes/.env is the authoritative config."
echo " /etc/profile.d/ is only used for interactive SSH sessions.)"
echo ""
+1030
View File
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,317 @@
#!/usr/bin/env bash
# ═══════════════════════════════════════════════════════════════════
# OpenClaw Patch: after_tool_call hook 注入 session messages
# ═══════════════════════════════════════════════════════════════════
# 用途:在 after_tool_call hookEvent 中注入 ctx.params.session?.messages
# 使 context-offload 等插件能在工具调用后访问完整历史消息列表。
#
# 兼容策略(按优先级尝试,首个成功即停止):
# 策略 1: AST-like — 搜索 hookEvent 对象中 durationMs 字段,在其后追加 messages
# 策略 2: 旧版 dispatch-*.js — 针对早期版本文件布局
# 策略 3: runAfterToolCall 锚点 — 在 hookEvent 关闭大括号前插入
# 策略 4: 通用 fallback — 基于 after_tool_call + durationMs 的宽松匹配
#
# 用法:
# bash openclaw-after-tool-call-messages.patch.sh
# bash openclaw-after-tool-call-messages.patch.sh /custom/path/to/openclaw
#
# 幂等:已 patch 过的文件会自动跳过,可安全重复执行。
# ═══════════════════════════════════════════════════════════════════
set -euo pipefail
RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'; CYAN='\033[0;36m'; NC='\033[0m'
info() { echo -e "${CYAN}[INFO]${NC} $*"; }
ok() { echo -e "${GREEN}[OK]${NC} $*"; }
warn() { echo -e "${YELLOW}[WARN]${NC} $*"; }
fail() { echo -e "${RED}[FAIL]${NC} $*"; exit 1; }
debug() { [[ "${DEBUG:-}" == "1" ]] && echo -e "${CYAN}[DEBUG]${NC} $*" || true; }
# ─── 定位 OpenClaw 安装目录 ──────────────────────────────────────
# Uses Node.js require.resolve to locate the package root — handles
# nvm, pnpm, npm, yarn, volta, and any other layout automatically.
_node_resolve_openclaw() {
node -e "
const {dirname, join} = require('path');
const {realpathSync, existsSync, readFileSync, statSync} = require('fs');
// Helper: walk up from a file/dir to find the openclaw package root
function walkUp(start) {
let dir = statSync(start).isDirectory() ? start : dirname(start);
for (let i = 0; i < 10; i++) {
const pj = join(dir, 'package.json');
if (existsSync(pj)) {
try {
const pkg = JSON.parse(readFileSync(pj, 'utf8'));
if (pkg.name === 'openclaw') return dir;
} catch {}
}
const parent = dirname(dir);
if (parent === dir) break;
dir = parent;
}
return null;
}
// Strategy 1: which openclaw → realpath → walk up
try {
const {execSync} = require('child_process');
const bin = execSync('which openclaw', {encoding:'utf8'}).trim();
const real = realpathSync(bin);
const found = walkUp(real);
if (found) { console.log(found); process.exit(0); }
// pnpm uses shell shims: the bin file is a script, not a symlink.
// Read the shim content to extract the real entry point path.
const content = readFileSync(bin, 'utf8');
// pnpm shim contains a line like: exec node \"/path/.../openclaw/dist/cli.js\"
// or: require(\"/path/.../openclaw/dist/cli.js\")
const m = content.match(/['\"]([^'\"]*openclaw[^'\"]*\\.(?:js|mjs))['\"]/) ||
content.match(/['\"]([^'\"]*openclaw[^'\"]*)['\"].*node/);
if (m) {
const shimTarget = realpathSync(m[1]);
const found2 = walkUp(shimTarget);
if (found2) { console.log(found2); process.exit(0); }
}
} catch {}
// Strategy 2: search common pnpm/npm global paths
const {execSync: exec2} = require('child_process');
const searchDirs = [
join(process.env.HOME || '/root', '.local/share/pnpm'),
join(process.env.HOME || '/root', '.local/node/lib/node_modules'),
'/usr/local/lib/node_modules',
'/usr/lib/node_modules',
];
for (const base of searchDirs) {
if (!existsSync(base)) continue;
try {
const out = exec2(
'find ' + JSON.stringify(base) + ' -maxdepth 8 -name package.json -path \"*/openclaw/package.json\" 2>/dev/null',
{encoding:'utf8', timeout: 5000}
).trim();
for (const line of out.split('\\n')) {
if (!line) continue;
try {
const pkg = JSON.parse(readFileSync(line, 'utf8'));
if (pkg.name === 'openclaw') { console.log(dirname(line)); process.exit(0); }
} catch {}
}
} catch {}
}
process.exit(1);
" 2>/dev/null
}
if [[ -n "${1:-}" ]]; then
OPENCLAW_DIR="$1"
elif OPENCLAW_DIR="$(_node_resolve_openclaw)"; then
debug "Node.js resolved openclaw → $OPENCLAW_DIR"
else
fail "找不到 OpenClaw 安装目录。请手动指定:\n bash $0 /path/to/openclaw"
fi
DIST_DIR="$OPENCLAW_DIR/dist"
if [[ ! -d "$DIST_DIR" ]]; then
fail "dist 目录不存在: $DIST_DIR"
fi
info "OpenClaw 目录: $OPENCLAW_DIR"
# ─── 检测 OpenClaw 版本 ──────────────────────────────────────────
VERSION=$(grep -oP '"version"\s*:\s*"\K[^"]+' "$OPENCLAW_DIR/package.json" 2>/dev/null || echo "unknown")
info "检测到 OpenClaw 版本: $VERSION"
# ─── 已 patch 检测 ───────────────────────────────────────────────
# 核心标记:hookEvent 内部 durationMs 之后紧跟 messages 注入
# 支持多种缩进格式(tab / 空格 / 混合)
INJECTION_CODE='messages: ctx.params.session?.messages'
INJECTION_CODE_ALT='messages:ctx.params.session?.messages'
is_already_patched() {
local f="$1"
# 方法 1: 精确检测 — durationMs 后面紧跟 messages 注入(允许任意空白)
if perl -0777 -ne 'exit(0) if /durationMs[,\s]*\n\s*messages\s*:\s*ctx\.params\.session\?\s*\.messages/; exit(1)' "$f" 2>/dev/null; then
return 0
fi
# 方法 2: 上下文检测 — after_tool_call hookEvent 对象内部(durationMs 附近)有 messages 注入
# 注意: 不能用全文件检测,因为 before_compaction 也有 messages: ctx.params.session.messages
if perl -0777 -ne 'exit(0) if /(?:hookEvent|hook_event)\s*=\s*\{[\s\S]{0,500}durationMs[\s\S]{0,100}messages\s*:\s*ctx\.params\.session/; exit(1)' "$f" 2>/dev/null; then
return 0
fi
return 1
}
verify_patch() {
local f="$1"
is_already_patched "$f"
}
# ─── 备份工具 ────────────────────────────────────────────────────
backup_file() {
local f="$1"
local bak="${f}.pre-offload-patch.bak"
if [[ ! -f "$bak" ]]; then
cp "$f" "$bak"
debug "备份: $bak"
fi
}
# ─── 查找所有候选文件 ────────────────────────────────────────────
# 收集所有包含 after_tool_call 的 JS 文件(不限子目录深度)
mapfile -t CANDIDATE_FILES < <(grep -rl 'after_tool_call' "$DIST_DIR" --include='*.js' 2>/dev/null || true)
if [[ ${#CANDIDATE_FILES[@]} -eq 0 ]]; then
warn "$DIST_DIR 下未找到包含 after_tool_call 的 JS 文件"
fi
info "找到 ${#CANDIDATE_FILES[@]} 个候选文件"
PATCHED=0
SKIPPED=0
FAILED=0
# ─── 对每个候选文件尝试多种策略 ──────────────────────────────────
for f in "${CANDIDATE_FILES[@]}"; do
fname="$(basename "$f")"
relpath="${f#$DIST_DIR/}"
# 已 patch → 跳过
if is_already_patched "$f"; then
warn "$relpath — 已经 patch 过,跳过"
((SKIPPED++)) || true
continue
fi
# 确认文件中有 durationMshookEvent 的标志字段)
if ! grep -q 'durationMs' "$f" 2>/dev/null; then
debug "$relpath — 不含 durationMs,非 patch 目标,跳过"
continue
fi
# 确认 durationMs 附近有 after_tool_call 上下文(避免误匹配 before_compaction 等)
if ! perl -0777 -ne 'exit(0) if /after_tool_call[\s\S]{0,2000}durationMs/; exit(1)' "$f" 2>/dev/null; then
debug "$relpath — durationMs 不在 after_tool_call 上下文中,跳过"
continue
fi
backup_file "$f"
applied=false
# ── 策略 1: hookEvent 对象中 durationMs 是最后一个字段 ────────
# 匹配: durationMs<换行><空白>};<换行><空白>hookRunnerAfter
# 或: durationMs<换行><空白>};<换行><空白>await ...hookRunner...afterToolCall
# 缩进用 \s+ 宽松匹配
if [[ "$applied" == "false" ]]; then
if perl -0777 -ne 'exit(0) if /durationMs\s*\n(\s*)\};\s*\n\s*(hookRunnerAfter|await\s+\S*hookRunner\S*\.runAfterToolCall|hookRunner\S*\.runAfterToolCall)/; exit(1)' "$f" 2>/dev/null; then
debug "$relpath — 命中策略1 (hookRunnerAfter 锚点)"
perl -0777 -i -pe 's/(durationMs)\s*\n(\s*\};\s*\n\s*(?:hookRunnerAfter|await\s+\S*hookRunner\S*\.runAfterToolCall|hookRunner\S*\.runAfterToolCall))/$1,\n\t\t\tmessages: ctx.params.session?.messages\n$2/' "$f"
if verify_patch "$f"; then
ok "[策略1] $relpath — patch 成功"
((PATCHED++)) || true
applied=true
fi
fi
fi
# ── 策略 2: 旧版 dispatch-*.js — durationMs 行末独占 ─────────
if [[ "$applied" == "false" ]]; then
if echo "$relpath" | grep -qP 'dispatch-.*\.js' 2>/dev/null; then
# 匹配行末独占的 durationMs(前面是空白)
if grep -qP '^\s+durationMs\s*$' "$f" 2>/dev/null; then
debug "$relpath — 命中策略2 (旧版 dispatch)"
sed -i -E 's/^(\s+)(durationMs)\s*$/\1\2,\n\1messages: ctx.params.session?.messages/' "$f"
if verify_patch "$f"; then
ok "[策略2] $relpath — patch 成功"
((PATCHED++)) || true
applied=true
fi
fi
fi
fi
# ── 策略 3: durationMs 后跟 }; 但无 hookRunnerAfter 锚点 ────
# 匹配: durationMs<换行><空白>}; hookEvent 闭合)
# 通过上下文(附近有 after_tool_call)确认是正确的对象
if [[ "$applied" == "false" ]]; then
# 找到包含 durationMs 且附近(±20行)有 after_tool_call 的代码区域
if perl -0777 -ne 'exit(0) if /after_tool_call[\s\S]{0,800}durationMs\s*\n(\s*)\};/; exit(1)' "$f" 2>/dev/null; then
debug "$relpath — 命中策略3 (durationMs→}; 邻近 after_tool_call)"
# 只替换 after_tool_call 上下文附近的 durationMs → };
perl -0777 -i -pe 's/(after_tool_call[\s\S]{0,800}durationMs)\s*\n(\s*\};)/$1,\n\t\t\tmessages: ctx.params.session?.messages\n$2/' "$f"
if verify_patch "$f"; then
ok "[策略3] $relpath — patch 成功"
((PATCHED++)) || true
applied=true
fi
fi
fi
# ── 策略 4: 通用 fallback — hookEvent 赋值中的 durationMs ────
# 匹配形如: const hookEvent = { ... durationMs ... }
# 或: hookEvent = { ... durationMs ... }
# 用 perl 找到包含 after_tool_call 和 durationMs 的对象字面量,在 durationMs 后插入
if [[ "$applied" == "false" ]]; then
# 极宽松: 在文件中找到 "durationMs" 后面最近的 "}" 或 "};" ,在中间插入
# 但限制在 after_tool_call 关键字附近 2000 字符内
if perl -0777 -ne 'exit(0) if /after_tool_call[\s\S]{0,2000}?(?:hookEvent|hook_event)[\s\S]{0,500}?durationMs/; exit(1)' "$f" 2>/dev/null; then
debug "$relpath — 命中策略4 (通用 fallback)"
# 在 durationMs 后追加 (仅首次匹配)
perl -0777 -i -pe '
my $done = 0;
s/(after_tool_call[\s\S]{0,2000}?(?:hookEvent|hook_event)[\s\S]{0,500}?durationMs)\s*\n(\s*)(\};)/
if (!$done) { $done = 1; "$1,\n$2\tmessages: ctx.params.session?.messages\n$2$3" }
else { "$1\n$2$3" }
/ge;
' "$f"
if verify_patch "$f"; then
ok "[策略4] $relpath — patch 成功"
((PATCHED++)) || true
applied=true
else
warn "[策略4] $relpath — patch 后验证失败,恢复备份"
cp "${f}.pre-offload-patch.bak" "$f"
fi
fi
fi
# ── 无策略命中 ───────────────────────────────────────────────
if [[ "$applied" == "false" ]]; then
debug "$relpath — 无策略命中"
((FAILED++)) || true
fi
done
# ─── 结果报告 ────────────────────────────────────────────────────
echo ""
echo -e "${GREEN}═══════════════════════════════════════════════════════${NC}"
echo -e "${GREEN} Patch 完成 (OpenClaw $VERSION)${NC}"
echo -e "${GREEN}═══════════════════════════════════════════════════════${NC}"
echo ""
echo -e " 成功: ${GREEN}${PATCHED}${NC} 跳过: ${YELLOW}${SKIPPED}${NC} 失败: ${RED}${FAILED}${NC}"
echo ""
if [[ $PATCHED -gt 0 ]]; then
echo -e " ${CYAN}重启 OpenClaw 后生效。${NC}"
echo -e " ${CYAN}备份文件: *.pre-offload-patch.bak${NC}"
elif [[ $SKIPPED -gt 0 && $FAILED -eq 0 ]]; then
echo -e " ${YELLOW}所有目标文件已 patch,无需重复操作。${NC}"
elif [[ $FAILED -gt 0 ]]; then
echo -e " ${RED}部分文件未能 patch。可能需要手动检查或更新 patch 脚本。${NC}"
echo -e " ${RED}提示:设置 DEBUG=1 运行以查看详细匹配过程:${NC}"
echo -e " ${RED} DEBUG=1 bash $0 $OPENCLAW_DIR${NC}"
else
echo -e " ${RED}未找到匹配的目标文件,请确认 OpenClaw 版本。${NC}"
echo -e " ${RED}提示:设置 DEBUG=1 运行以查看详细匹配过程:${NC}"
echo -e " ${RED} DEBUG=1 bash $0 $OPENCLAW_DIR${NC}"
fi
echo ""
# ─── 退出码 ───────────────────────────────────────────────────────
# 0: 成功(至少有一个文件 patch 成功或已跳过)
# 1: 失败(无任何 patch 成功且无已跳过的文件)
if [[ $PATCHED -gt 0 || $SKIPPED -gt 0 ]]; then
exit 0
else
exit 1
fi
+340
View File
@@ -0,0 +1,340 @@
#!/usr/bin/env bash
# ═══════════════════════════════════════════════════════════════════
# setup-offload.sh — Offload 功能一键开启/关闭
# ═══════════════════════════════════════════════════════════════════
#
# 用法:
# bash setup-offload.sh --enable --user-id <userId> --backend-url <url> [--backend-api-key <key>]
# bash setup-offload.sh --disable
# bash setup-offload.sh --status
#
# 开启流程:
# 1. 前置检查(openclaw.json 存在、openclaw 已安装)
# 2. Patch 校验 & 执行(after_tool_call messages 注入)— 失败则终止
# 3. 设置 plugins.slots.contextEngine
# 4. 设置 offload.enabled + backendUrl + userId [+ backendApiKey]
# 5. 设置 compaction.mode = safeguard
#
# 关闭流程:
# 1. 设置 offload.enabled = false
# 2. 删除 plugins.slots.contextEngine(清理 slot 占用)
# ═══════════════════════════════════════════════════════════════════
set -euo pipefail
# ── 颜色 ──
RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'
CYAN='\033[0;36m'; BOLD='\033[1m'; NC='\033[0m'
info() { echo -e "${CYAN}[INFO]${NC} $*"; }
ok() { echo -e "${GREEN}[OK]${NC} $*"; }
warn() { echo -e "${YELLOW}[WARN]${NC} $*"; }
fail() { echo -e "${RED}[FAIL]${NC} $*" >&2; exit 1; }
# ── 常量 ──
OPENCLAW_JSON="${HOME}/.openclaw/openclaw.json"
PLUGIN_ID="memory-tencentdb"
CONTEXT_ENGINE_ID="openclaw-context-offload"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PATCH_SCRIPT="${SCRIPT_DIR}/openclaw-after-tool-call-messages.patch.sh"
# ── 参数解析 ──
MODE=""
USER_ID=""
BACKEND_URL=""
BACKEND_API_KEY=""
while [[ $# -gt 0 ]]; do
case "$1" in
--enable) MODE="enable"; shift ;;
--disable) MODE="disable"; shift ;;
--status) MODE="status"; shift ;;
--user-id) USER_ID="$2"; shift 2 ;;
--backend-url) BACKEND_URL="$2"; shift 2 ;;
--backend-api-key) BACKEND_API_KEY="$2"; shift 2 ;;
-h|--help)
echo "用法:"
echo " bash setup-offload.sh --enable --user-id <userId> --backend-url <url> [--backend-api-key <key>]"
echo " bash setup-offload.sh --disable"
echo " bash setup-offload.sh --status"
echo ""
echo "参数:"
echo " --user-id (必填) 用户 ID"
echo " --backend-url (必填) offload 后端地址,如 http://1.2.3.4:8080"
echo " --backend-api-key (可选) 后端 API 认证 token"
exit 0
;;
*) fail "未知参数: $1" ;;
esac
done
[[ -z "$MODE" ]] && fail "请指定模式: --enable / --disable / --status"
# ═══════════════════════════════════════════════════════════════════
# 公共函数
# ═══════════════════════════════════════════════════════════════════
check_openclaw_json() {
if [[ ! -f "$OPENCLAW_JSON" ]]; then
fail "openclaw.json 不存在: $OPENCLAW_JSON"
fi
# 验证 JSON 格式
python3 -c "import json; json.load(open('$OPENCLAW_JSON'))" 2>/dev/null \
|| fail "openclaw.json 格式错误"
}
backup_config() {
local bak="${OPENCLAW_JSON}.bak.$(date +%Y%m%d_%H%M%S)"
cp "$OPENCLAW_JSON" "$bak"
info "配置已备份: $bak"
}
# ═══════════════════════════════════════════════════════════════════
# --status: 显示当前配置状态
# ═══════════════════════════════════════════════════════════════════
show_status() {
check_openclaw_json
python3 -c "
import json
with open('$OPENCLAW_JSON') as f:
cfg = json.load(f)
# Context Engine Slot
slot = cfg.get('plugins', {}).get('slots', {}).get('contextEngine', '(未设置)')
print(f' Context Engine Slot: {slot}')
# Offload config
offload = cfg.get('plugins', {}).get('entries', {}).get('$PLUGIN_ID', {}).get('config', {}).get('offload', {})
enabled = offload.get('enabled', False)
backend = offload.get('backendUrl', '(未设置)')
user_id = offload.get('userId', '(未设置)')
api_key = offload.get('backendApiKey', '')
timeout = offload.get('backendTimeoutMs', '(默认)')
mild = offload.get('mildOffloadRatio', '(默认 0.5)')
agg = offload.get('aggressiveCompressRatio', '(默认 0.85)')
status_icon = '✅ 已开启' if enabled else '❌ 已关闭'
api_key_display = f'{api_key[:8]}...' if api_key and len(api_key) > 8 else (api_key or '(未设置)')
print(f' Offload 状态: {status_icon}')
print(f' Backend URL: {backend}')
print(f' Backend Key: {api_key_display}')
print(f' User ID: {user_id}')
print(f' Timeout: {timeout}ms')
print(f' Mild Ratio: {mild}')
print(f' Aggressive: {agg}')
# Compaction mode
compaction = cfg.get('agents', {}).get('defaults', {}).get('compaction', {}).get('mode', '(未设置)')
print(f' Compaction: {compaction}')
"
}
# ═══════════════════════════════════════════════════════════════════
# --enable: 开启 offload
# ═══════════════════════════════════════════════════════════════════
enable_offload() {
# 参数校验
[[ -z "$USER_ID" ]] && fail "缺少 --user-id 参数"
[[ -z "$BACKEND_URL" ]] && fail "缺少 --backend-url 参数"
# URL 格式基本校验
if [[ ! "$BACKEND_URL" =~ ^https?:// ]]; then
fail "backendUrl 格式错误,应以 http:// 或 https:// 开头: $BACKEND_URL"
fi
check_openclaw_json
backup_config
echo ""
info "${BOLD}[1/4] Patch 校验${NC}"
# ── Step 1: 调用 patch 脚本(幂等,已 patch 过会跳过) ──
# patch 脚本自带精确的幂等检测,通过退出码判断结果:
# 0 = 成功(新 patch 或已跳过)
# 1 = 失败(无法 patch
if [[ -f "$PATCH_SCRIPT" ]]; then
info "执行 patch 脚本..."
local patch_exit=0
local patch_output
patch_output=$(bash "$PATCH_SCRIPT" 2>&1) || patch_exit=$?
# 显示 patch 脚本输出(缩进)
while IFS= read -r line; do
echo " $line"
done <<< "$patch_output"
if [[ $patch_exit -eq 0 ]]; then
ok "Patch 校验通过"
else
echo ""
echo -e "${RED}═══════════════════════════════════════════════════${NC}"
echo -e "${RED} ❌ Patch 失败(退出码: $patch_exit${NC}"
echo -e "${RED}═══════════════════════════════════════════════════${NC}"
echo ""
echo -e " ${RED}after_tool_call hook 无法获取 session messages${NC}"
echo -e " ${RED}offload L1/L3 压缩将无法正常工作。${NC}"
echo ""
echo -e " ${CYAN}排查步骤:${NC}"
echo -e " 1. DEBUG=1 bash $PATCH_SCRIPT"
echo -e " 2. 检查 openclaw 版本是否兼容"
echo ""
exit 2
fi
else
echo -e "${RED}[FAIL]${NC} Patch 脚本不存在: $PATCH_SCRIPT" >&2
echo -e " ${RED}offload 功能依赖此 patch,终止开启流程。${NC}" >&2
exit 2
fi
# ── Step 2: 设置 context engine slot ──
echo ""
info "${BOLD}[2/4] 设置 Context Engine Slot${NC}"
python3 -c "
import json
with open('$OPENCLAW_JSON') as f:
cfg = json.load(f)
# 确保 plugins.slots 存在
cfg.setdefault('plugins', {}).setdefault('slots', {})
cfg['plugins']['slots']['contextEngine'] = '$CONTEXT_ENGINE_ID'
print(' slots.contextEngine = $CONTEXT_ENGINE_ID')
with open('$OPENCLAW_JSON', 'w') as f:
json.dump(cfg, f, indent=2, ensure_ascii=False)
f.write('\n')
"
ok "Context Engine Slot 已设置"
# ── Step 3: 设置 offload 配置 ──
echo ""
info "${BOLD}[3/4] 设置 Offload 配置${NC}"
python3 -c "
import json
with open('$OPENCLAW_JSON') as f:
cfg = json.load(f)
# 确保路径存在
entry = cfg.setdefault('plugins', {}).setdefault('entries', {}).setdefault('$PLUGIN_ID', {})
config = entry.setdefault('config', {})
offload = config.setdefault('offload', {})
# 设置必要配置
offload['enabled'] = True
offload['backendUrl'] = '$BACKEND_URL'
offload['userId'] = '$USER_ID'
offload.setdefault('backendTimeoutMs', 120000)
api_key = '$BACKEND_API_KEY'
if api_key:
offload['backendApiKey'] = api_key
print(f' offload.backendApiKey = {api_key[:8]}...' if len(api_key) > 8 else f' offload.backendApiKey = {api_key}')
elif 'backendApiKey' in offload:
del offload['backendApiKey']
print(' offload.enabled = true')
print(' offload.backendUrl = $BACKEND_URL')
print(' offload.userId = $USER_ID')
print(f' offload.backendTimeoutMs = {offload[\"backendTimeoutMs\"]}')
with open('$OPENCLAW_JSON', 'w') as f:
json.dump(cfg, f, indent=2, ensure_ascii=False)
f.write('\n')
"
ok "Offload 配置已设置"
# ── Step 4: 设置 compaction mode ──
echo ""
info "${BOLD}[4/4] 设置 Compaction Mode${NC}"
python3 -c "
import json
with open('$OPENCLAW_JSON') as f:
cfg = json.load(f)
defaults = cfg.setdefault('agents', {}).setdefault('defaults', {})
compaction = defaults.setdefault('compaction', {})
old_mode = compaction.get('mode', '(未设置)')
compaction['mode'] = 'safeguard'
print(f' compaction.mode: {old_mode} → safeguard')
with open('$OPENCLAW_JSON', 'w') as f:
json.dump(cfg, f, indent=2, ensure_ascii=False)
f.write('\n')
"
ok "Compaction mode 已设置为 safeguard"
# ── 完成 ──
echo ""
echo -e "${GREEN}═══════════════════════════════════════════════════${NC}"
echo -e "${GREEN} ✅ Offload 已开启${NC}"
echo -e "${GREEN}═══════════════════════════════════════════════════${NC}"
echo ""
show_status
echo ""
echo -e " ${CYAN}提示: 需要重启 gateway 才能生效${NC}"
echo -e " ${CYAN} bash install-plugin.sh --restart${NC}"
}
# ═══════════════════════════════════════════════════════════════════
# --disable: 关闭 offload
# ═══════════════════════════════════════════════════════════════════
disable_offload() {
check_openclaw_json
backup_config
python3 -c "
import json
with open('$OPENCLAW_JSON') as f:
cfg = json.load(f)
# 关闭 offload.enabled(用 setdefault 确保路径存在,修改能回写到 cfg)
entry = cfg.setdefault('plugins', {}).setdefault('entries', {}).setdefault('$PLUGIN_ID', {})
config = entry.setdefault('config', {})
offload = config.setdefault('offload', {})
offload['enabled'] = False
print(' offload.enabled = false')
# 移除 contextEngine slot
plugins = cfg.get('plugins', {})
slots = plugins.get('slots', {})
if 'contextEngine' in slots:
del slots['contextEngine']
print(' slots.contextEngine → 已删除')
# 如果 slots 变空了,也一并移除 slots 键
if not slots and 'slots' in plugins:
del plugins['slots']
print(' plugins.slots → 已清理(空对象)')
else:
print(' slots.contextEngine → 无需删除(不存在)')
with open('$OPENCLAW_JSON', 'w') as f:
json.dump(cfg, f, indent=2, ensure_ascii=False)
f.write('\n')
"
echo ""
echo -e "${YELLOW}═══════════════════════════════════════════════════${NC}"
echo -e "${YELLOW} ❌ Offload 已关闭${NC}"
echo -e "${YELLOW}═══════════════════════════════════════════════════${NC}"
echo ""
echo -e " ${CYAN}提示: 需要重启 gateway 才能生效${NC}"
echo -e " ${CYAN} bash install-plugin.sh --restart${NC}"
}
# ═══════════════════════════════════════════════════════════════════
# 主入口
# ═══════════════════════════════════════════════════════════════════
case "$MODE" in
enable) enable_offload ;;
disable) disable_offload ;;
status)
echo ""
info "${BOLD}Offload 配置状态${NC}"
show_status
;;
esac
+319
View File
@@ -0,0 +1,319 @@
/**
* TDAI Memory Test Script
*
* 测试 TDAI 记忆功能是否正常工作。
* 启动 Gateway,然后发送测试请求验证记忆捕获和召回。
*/
import { spawn } from "node:child_process";
import { createServer } from "node:http";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import { EventEmitter } from "node:events";
import { tmpdir } from "node:os";
import { mkdir, rm, writeFile } from "node:fs/promises";
const __dirname = dirname(fileURLToPath(import.meta.url));
const PROJECT_ROOT = join(__dirname, "..");
// 测试配置
const TEST_CONFIG = {
server: {
// Use 8422 by default to avoid clashing with the hermes sidecar on 8420.
port: Number(process.env.TDAI_TEST_PORT || 8422),
host: "127.0.0.1",
},
data: {
baseDir: join(tmpdir(), "tdai-memory-test"),
},
llm: {
baseUrl: process.env.TDAI_LLM_BASE_URL || "https://vdbteam.openai.azure.com/openai/v1",
apiKey: process.env.TDAI_LLM_API_KEY || "",
model: process.env.TDAI_LLM_MODEL || "gpt-5.2-chat",
maxTokens: 4096,
timeoutMs: 120000,
},
};
// 用于接收 Gateway 回调的本地服务器
const callbackServer = createServer((req, res) => {
console.log(`[Callback] ${req.method} ${req.url}`);
res.writeHead(200, { "Content-Type": "application/json" });
res.end(JSON.stringify({ status: "ok" }));
});
let gatewayProcess = null;
async function ensureDataDir() {
await mkdir(TEST_CONFIG.data.baseDir, { recursive: true });
const configPath = join(TEST_CONFIG.data.baseDir, "tdai-gateway.yaml");
const configContent = `# TDAI Gateway Test Config
server:
port: ${TEST_CONFIG.server.port}
host: ${TEST_CONFIG.server.host}
data:
baseDir: ${TEST_CONFIG.data.baseDir}
llm:
baseUrl: ${TEST_CONFIG.llm.baseUrl}
apiKey: ${TEST_CONFIG.llm.apiKey || "sk-test-key"}
model: ${TEST_CONFIG.llm.model}
maxTokens: ${TEST_CONFIG.llm.maxTokens}
timeoutMs: ${TEST_CONFIG.llm.timeoutMs}
memory:
capture:
enabled: true
l0l1RetentionDays: 7
extraction:
enabled: true
enableDedup: true
recall:
enabled: true
maxResults: 5
scoreThreshold: 0.1
strategy: hybrid
pipeline:
everyNConversations: 3
enableWarmup: true
`;
await writeFile(configPath, configContent);
console.log(`[Config] Written to ${configPath}`);
}
function startCallbackServer() {
const port = 18420;
return new Promise((resolve, reject) => {
callbackServer.listen(port, "127.0.0.1", () => {
console.log(`[CallbackServer] Started on port ${port}`);
resolve(port);
});
callbackServer.on("error", reject);
});
}
function startGateway(callbackPort) {
return new Promise((resolve, reject) => {
const configPath = join(TEST_CONFIG.data.baseDir, "tdai-gateway.yaml");
// Launch the TS source directly via npx/tsx (no dist build step required).
const gatewayArgs = [
"npx",
"--yes",
"tsx",
"src/gateway/server.ts",
];
console.log(`[Gateway] Starting with args:`, gatewayArgs);
const env = {
...process.env,
TDAI_GATEWAY_PORT: String(TEST_CONFIG.server.port),
TDAI_GATEWAY_HOST: TEST_CONFIG.server.host,
TDAI_DATA_DIR: TEST_CONFIG.data.baseDir,
TDAI_LLM_BASE_URL: TEST_CONFIG.llm.baseUrl,
TDAI_LLM_API_KEY: TEST_CONFIG.llm.apiKey,
TDAI_LLM_MODEL: TEST_CONFIG.llm.model,
TDAI_CALLBACK_PORT: String(callbackPort),
};
gatewayProcess = spawn(gatewayArgs[0], gatewayArgs.slice(1), {
cwd: PROJECT_ROOT,
env,
stdio: ["ignore", "pipe", "pipe"],
});
gatewayProcess.stdout?.on("data", (data) => {
console.log(`[Gateway] ${data.toString().trim()}`);
});
gatewayProcess.stderr?.on("data", (data) => {
console.error(`[Gateway Error] ${data.toString().trim()}`);
});
let healthCheckCount = 0;
const maxHealthChecks = 30;
const checkInterval = 1000;
const checkHealth = () => {
healthCheckCount++;
const req = new Request(`http://${TEST_CONFIG.server.host}:${TEST_CONFIG.server.port}/health`);
fetch(req)
.then((res) => {
if (res.ok) {
resolve({ process: gatewayProcess, port: TEST_CONFIG.server.port });
return;
}
})
.catch(() => {});
if (healthCheckCount >= maxHealthChecks) {
reject(new Error("Gateway failed to start"));
}
setTimeout(checkHealth, checkInterval);
};
setTimeout(checkHealth, checkInterval);
gatewayProcess.on("error", (err) => {
reject(new Error(`Failed to start gateway: ${err.message}`));
});
});
}
async function testHealthCheck(gatewayPort) {
console.log("\n[Test] Health check...");
try {
const res = await fetch(`http://127.0.0.1:${gatewayPort}/health`);
const data = await res.json();
console.log(`[OK] Health check passed:`, JSON.stringify(data, null, 2));
return true;
} catch (err) {
console.error(`[FAIL] Health check failed:`, err.message);
return false;
}
}
async function testCapture(gatewayPort) {
console.log("\n[Test] Capture conversation...");
try {
const res = await fetch(`http://127.0.0.1:${gatewayPort}/capture`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
user_content: "你好,我是一个测试用户。我喜欢编程和咖啡。",
assistant_content: "你好!很高兴认识你。编程和咖啡都是很棒的爱好!",
session_key: "test-session-001",
session_id: "test-session-id-001",
user_id: "test-user",
}),
});
const data = await res.json();
console.log(`[OK] Capture response:`, JSON.stringify(data, null, 2));
return true;
} catch (err) {
console.error(`[FAIL] Capture failed:`, err.message);
return false;
}
}
async function testRecall(gatewayPort) {
console.log("\n[Test] Recall memories...");
try {
const res = await fetch(`http://127.0.0.1:${gatewayPort}/recall`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
query: "用户的爱好是什么?",
session_key: "test-session-001",
user_id: "test-user",
}),
});
const data = await res.json();
console.log(`[OK] Recall response:`, JSON.stringify(data, null, 2));
return true;
} catch (err) {
console.error(`[FAIL] Recall failed:`, err.message);
return false;
}
}
async function testSearchMemories(gatewayPort) {
console.log("\n[Test] Search memories...");
try {
const res = await fetch(`http://127.0.0.1:${gatewayPort}/search/memories`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
query: "编程",
limit: 5,
type: "episode",
}),
});
const data = await res.json();
console.log(`[OK] Search memories response:`, JSON.stringify(data, null, 2));
return true;
} catch (err) {
console.error(`[FAIL] Search memories failed:`, err.message);
return false;
}
}
async function stopGateway() {
if (gatewayProcess) {
console.log("\n[Cleanup] Stopping gateway...");
gatewayProcess.kill("SIGTERM");
gatewayProcess = null;
}
if (callbackServer.listening) {
callbackServer.close();
}
}
async function cleanup() {
await stopGateway();
// 不删除测试数据目录,保留用于调试
console.log("[Cleanup] Test data preserved at:", TEST_CONFIG.data.baseDir);
}
async function main() {
console.log("=".repeat(60));
console.log("TDAI Memory Test");
console.log("=".repeat(60));
const results = {
health: false,
capture: false,
recall: false,
search: false,
};
try {
// 1. 确保数据目录存在
await ensureDataDir();
// 2. 启动回调服务器
const callbackPort = await startCallbackServer();
// 3. 启动 Gateway
const { port: gatewayPort } = await startGateway(callbackPort);
// 等待 Gateway 完全启动
await new Promise((resolve) => setTimeout(resolve, 2000));
// 4. 运行测试
results.health = await testHealthCheck(gatewayPort);
if (results.health) {
// 等待一下让 Gateway 初始化
await new Promise((resolve) => setTimeout(resolve, 1000));
results.capture = await testCapture(gatewayPort);
if (results.capture) {
// 等待提取完成
await new Promise((resolve) => setTimeout(resolve, 3000));
results.recall = await testRecall(gatewayPort);
results.search = await testSearchMemories(gatewayPort);
}
}
} catch (err) {
console.error("\n[Test] Error:", err.message);
} finally {
await cleanup();
// 打印测试结果
console.log("\n" + "=".repeat(60));
console.log("Test Results:");
console.log("=".repeat(60));
console.log(` Health Check: ${results.health ? "PASS" : "FAIL"}`);
console.log(` Capture: ${results.capture ? "PASS" : "FAIL"}`);
console.log(` Recall: ${results.recall ? "PASS" : "FAIL"}`);
console.log(` Search: ${results.search ? "PASS" : "FAIL"}`);
const passed = Object.values(results).filter(Boolean).length;
console.log(`\nTotal: ${passed}/4 passed`);
}
}
main().catch(console.error);