mirror of
https://github.com/TencentCloud/TencentDB-Agent-Memory
synced 2026-07-10 20:34:30 +00:00
feat: release v1.0.0
This commit is contained in:
@@ -4,6 +4,71 @@
|
||||
|
||||
---
|
||||
|
||||
## [1.0.0] - 2026-06-11
|
||||
|
||||
> **正式版发布**:从 OpenClaw 专属插件演进为**面向所有 Agent 的通用记忆服务**。完整的 Gateway 独立服务 + v2 HTTP API + 官方 TypeScript / Python SDK,任何 Agent 框架均可接入完整的多层记忆与上下文压缩能力。
|
||||
|
||||
### ⚠️ Breaking Changes
|
||||
|
||||
- **客户端/服务端架构拆分**:记忆引擎从 OpenClaw 嵌入式插件拆分为独立 Gateway 服务进程,部署方式与接入方式发生变化。
|
||||
- **配置结构变更**:插件配置结构扁平化重构,原 `gateway` 字段迁移为 `server` 嵌套。
|
||||
- **插件入口模式变更**:支持 `local`(进程内本地运行,默认)和 `client`(连接外部 Memory Gateway)两种接入模式。
|
||||
|
||||
### ✨ 新功能
|
||||
|
||||
#### 独立 Gateway 服务(v2 API)
|
||||
|
||||
记忆能力不再绑定 OpenClaw 宿主,以独立服务形式运行,通过 v2 HTTP API 为任意 Agent 提供记忆读写与管线管理:
|
||||
|
||||
- **完整 v2 API**:14 条标准路由覆盖记忆 CRUD、原子更新、场景索引、管线状态查询等全部操作。
|
||||
- **管线状态查询(`/v2/pipeline/status`)**:实时获取 L1/L2/L3 各阶段运行状态与进度。
|
||||
- **实例生命周期管理(`/v2/instance/destroy`)**:支持外部系统主动创建/销毁记忆实例。
|
||||
- **可选 Bearer 鉴权 + CORS 白名单**:保护对外暴露的 API 安全。
|
||||
- **请求体校验**:强制 1 MiB 上限,防止异常请求。
|
||||
|
||||
#### 官方 SDK
|
||||
|
||||
- **TypeScript SDK 1.0.0**(`@tencentdb-agent-memory/memory-sdk-ts`):类型安全,覆盖全部 v2 API,npm 安装即用。
|
||||
- **Python SDK**(`tencentdb-agent-memory-sdk-python`):pip wheel 安装,同步/异步双模式,覆盖全部 v2 API。
|
||||
|
||||
#### 通用 Agent 框架适配
|
||||
|
||||
- **OpenClaw 插件适配**:支持 `local`(进程内本地运行,默认)和 `client`(连接外部 Memory Gateway)两种接入模式。`local` 模式保持原有体验。
|
||||
- **Hermes Agent 适配**:`memory_tencentdb_v2` adapter,支持 Hermes 框架多租户场景。
|
||||
- **通用接入**:任何能发 HTTP 请求的 Agent(LangChain、AutoGPT、自研框架等)均可通过 SDK 或裸 API 接入。
|
||||
|
||||
#### 可观测性
|
||||
|
||||
- **OpenTelemetry 全链路 Trace**:支持 OTLP 协议上报,可对接 Jaeger / Grafana Tempo 等后端。
|
||||
- **Langfuse 集成**:仅转发 LLM 相关 span,适合评估记忆提取质量。
|
||||
- **管线评测指标**:L1 提取率、去重决策分布、各阶段 token 消耗、recall 延迟等。
|
||||
|
||||
#### 部署
|
||||
|
||||
- **Standalone Docker 镜像**:单容器即可运行完整记忆服务,无需外部依赖。
|
||||
- **Hermes + Memory All-in-One 镜像**:Agent + 记忆服务一体化部署方案。
|
||||
- **`memory-tencentdb-ctl` CLI**:运维命令行工具,支持配置管理、存储引擎切换等操作。
|
||||
- **v2 安装器脚本**:一键配置插件适配层。
|
||||
|
||||
### 📖 文档
|
||||
|
||||
- SDK 接入指南(TypeScript / Python 示例)。
|
||||
- Gateway 独立部署文档。
|
||||
- OpenClaw / Hermes 插件安装说明。
|
||||
- Docker 部署与配置参考。
|
||||
|
||||
---
|
||||
|
||||
<details>
|
||||
<summary>1.0.0 预发布版本</summary>
|
||||
|
||||
## [1.0.0-beta.2] - 2026-06-05
|
||||
|
||||
### ✨ 新功能
|
||||
|
||||
- 统一插件入口,支持 local(进程内本地运行,默认)和 client(连接外部 Memory Gateway)两种接入模式。
|
||||
- 新增 Offload Server V2:L1/L1.5/L2 异步执行器、L3 压缩处理器、MMD 注入;Gateway 集成 /v2/offload/* 路由;
|
||||
|
||||
## [1.0.0-beta.1] - 2026-05-29
|
||||
|
||||
> **架构大版本**:从 OpenClaw 专属插件演进为**面向所有 Agent 的通用记忆服务**。核心能力通过独立 Gateway 服务 + 标准化 v2 HTTP API 对外暴露,配套官方 TypeScript / Python SDK,任何 Agent 框架均可接入完整的多层记忆管线。
|
||||
@@ -56,6 +121,8 @@
|
||||
- OpenClaw / Hermes 插件安装说明。
|
||||
- Docker 部署与配置参考。
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## [0.3.6] - 2026-05-27
|
||||
|
||||
@@ -121,6 +121,53 @@ docker pull agentmemory/hermes-memory:1.0.0-beta
|
||||
docker pull agentmemory/openclaw-memory:1.0.0-beta
|
||||
```
|
||||
|
||||
### 1.2 Zero-config to enable
|
||||
|
||||
Defaults to a local `SQLite + sqlite-vec` backend.
|
||||
|
||||
```jsonc
|
||||
// ~/.openclaw/openclaw.json
|
||||
{
|
||||
"memory-tencentdb": {
|
||||
"enabled": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Once enabled, TencentDB Agent Memory automatically handles conversation capture, memory extraction, scene aggregation, persona generation, and recall before the next turn.
|
||||
|
||||
### 1.3 Enable short-term compression (optional, requires version ≥ 0.3.4)
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"memory-tencentdb": {
|
||||
"config": {
|
||||
"offload": {
|
||||
"enabled": true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Step 1 — Register the slot in your plugin config
|
||||
|
||||
Add the `slots` field so OpenClaw routes context-offload requests to this plugin:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"plugins": {
|
||||
"slots": {
|
||||
"contextEngine": "memory-tencentdb"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Step 2 — Apply the runtime patch
|
||||
|
||||
For the best results, run the patch script below. It hooks `after-tool-call` messages so they can be offloaded and recovered correctly:
|
||||
|
||||
#### Hermes + Memory (standalone)
|
||||
|
||||
```bash
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
|
||||
<div align="center">
|
||||
|
||||
<img src="./assets/images/logo.png" alt="TencentDB Agent Memory" width="880" />
|
||||
@@ -126,6 +127,54 @@ docker pull agentmemory/hermes-memory:1.0.0-beta
|
||||
docker pull agentmemory/openclaw-memory:1.0.0-beta
|
||||
```
|
||||
|
||||
### 1.2 零配置启用
|
||||
|
||||
默认使用本地 `SQLite + sqlite-vec` 后端。
|
||||
|
||||
```jsonc
|
||||
// ~/.openclaw/openclaw.json
|
||||
{
|
||||
"memory-tencentdb": {
|
||||
"enabled": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
启用后,TencentDB Agent Memory 会自动完成对话录制、记忆提取、场景归纳、用户画像生成和下一轮对话前召回。
|
||||
|
||||
|
||||
### 1.3 启用短期记忆压缩(可选,要求版本 ≥ 0.3.4)
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"memory-tencentdb": {
|
||||
"config": {
|
||||
"offload": {
|
||||
"enabled": true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### 步骤 1 —— 在插件配置中注册 slot
|
||||
|
||||
在 `slots` 字段中声明 `contextEngine`,让 OpenClaw 把上下文卸载请求路由到本插件:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"plugins": {
|
||||
"slots": {
|
||||
"contextEngine": "memory-tencentdb"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### 步骤 2 —— 执行 patch 脚本
|
||||
|
||||
为保证最佳效果,请执行以下 patch 脚本。该脚本会注入 `after-tool-call` 消息钩子,让工具调用结果能被正确卸载与回溯:
|
||||
|
||||
#### Hermes + Memory(standalone)
|
||||
|
||||
```bash
|
||||
|
||||
@@ -971,11 +971,19 @@ class MemoryTencentdbProvider(MemoryProvider):
|
||||
except Exception as e:
|
||||
logger.debug("memory-tencentdb session end failed: %s", e)
|
||||
|
||||
# Note: do NOT shut down the supervisor/Gateway here — it may serve
|
||||
# other sessions. The Gateway manages its own lifecycle.
|
||||
# We *do* drop our reference to the supervisor so any in-flight
|
||||
# _try_recover_gateway() call sees self._supervisor is None and
|
||||
# bails out instead of resurrecting a released provider.
|
||||
# Stop only the Gateway process this supervisor spawned. If the
|
||||
# provider merely attached to an already-running external Gateway,
|
||||
# GatewaySupervisor.shutdown() is a no-op because _process is None.
|
||||
supervisor = self._supervisor
|
||||
if supervisor is not None:
|
||||
try:
|
||||
supervisor.shutdown()
|
||||
except Exception as e:
|
||||
logger.debug("memory-tencentdb supervisor shutdown failed: %s", e)
|
||||
|
||||
# Drop our reference so any in-flight _try_recover_gateway() call sees
|
||||
# self._supervisor is None and bails out instead of resurrecting a
|
||||
# released provider.
|
||||
self._client = None
|
||||
self._gateway_available = False
|
||||
self._initialized = False
|
||||
|
||||
@@ -8,12 +8,22 @@ On shutdown(), sends a flush signal and waits for clean exit.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import shlex
|
||||
import signal
|
||||
import subprocess
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
from typing import IO, Optional
|
||||
from typing import Dict, IO, Iterator, Optional
|
||||
|
||||
try:
|
||||
import fcntl
|
||||
except ImportError: # pragma: no cover - Windows fallback
|
||||
fcntl = None # type: ignore[assignment]
|
||||
|
||||
from .client import MemoryTencentdbSdkClient
|
||||
|
||||
@@ -31,6 +41,39 @@ HEALTH_CHECK_RETRIES = 3 # retries for is_running check
|
||||
# Log file rotation parameters
|
||||
LOG_TAIL_BYTES_ON_CRASH = 2048 # bytes of stderr log to surface on startup crash
|
||||
|
||||
# Startup single-flight state. The thread lock prevents multiple supervisor
|
||||
# instances in the same Python process from spawning the same Gateway
|
||||
# concurrently; the optional fcntl lock extends the guard across processes.
|
||||
_START_LOCKS: Dict[str, threading.Lock] = {}
|
||||
_START_LOCKS_GUARD = threading.Lock()
|
||||
|
||||
|
||||
def _lock_key(host: str, port: int) -> str:
|
||||
safe_host = re.sub(r"[^A-Za-z0-9_.-]", "_", host)
|
||||
return f"{safe_host}-{port}"
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def _startup_singleflight(host: str, port: int) -> Iterator[None]:
|
||||
key = _lock_key(host, port)
|
||||
with _START_LOCKS_GUARD:
|
||||
thread_lock = _START_LOCKS.setdefault(key, threading.Lock())
|
||||
|
||||
lock_path = os.path.join(tempfile.gettempdir(), f"memory-tencentdb-gateway-{key}.lock")
|
||||
with thread_lock:
|
||||
lock_file = None
|
||||
try:
|
||||
if fcntl is not None:
|
||||
lock_file = open(lock_path, "a", encoding="utf-8")
|
||||
fcntl.flock(lock_file.fileno(), fcntl.LOCK_EX)
|
||||
yield
|
||||
finally:
|
||||
if lock_file is not None:
|
||||
try:
|
||||
fcntl.flock(lock_file.fileno(), fcntl.LOCK_UN)
|
||||
finally:
|
||||
lock_file.close()
|
||||
|
||||
|
||||
class GatewaySupervisor:
|
||||
"""Manages the memory-tencentdb Gateway sidecar lifecycle."""
|
||||
@@ -76,6 +119,7 @@ class GatewaySupervisor:
|
||||
self._stdout_log: Optional[IO[bytes]] = None
|
||||
self._stderr_log: Optional[IO[bytes]] = None
|
||||
self._stderr_log_path: Optional[str] = None
|
||||
self._shutdown_requested = False
|
||||
|
||||
# Resolve Gateway command
|
||||
# Priority: explicit arg > MEMORY_TENCENTDB_GATEWAY_CMD env
|
||||
@@ -146,77 +190,96 @@ class GatewaySupervisor:
|
||||
logger.info("memory-tencentdb Gateway already running at %s", self._base_url)
|
||||
return True
|
||||
|
||||
# If we previously spawned a child and it has since died, drop the
|
||||
# stale Popen handle so the new spawn below isn't shadowed by a
|
||||
# zombie reference. Without this, a crashed-then-respawned Gateway
|
||||
# would keep ``self._process`` pointing at the dead PID forever and
|
||||
# ``is_process_alive()`` would mislead the watchdog.
|
||||
self._reap_dead_process()
|
||||
|
||||
# Try to start the Gateway
|
||||
if not self._gateway_cmd:
|
||||
logger.warning(
|
||||
"memory-tencentdb Gateway is not running and no gateway command configured. "
|
||||
"Set MEMORY_TENCENTDB_GATEWAY_CMD environment variable or pass gateway_cmd to supervisor. "
|
||||
"memory-tencentdb memory will be unavailable."
|
||||
)
|
||||
return False
|
||||
|
||||
logger.info("Starting memory-tencentdb Gateway: %s", self._gateway_cmd)
|
||||
|
||||
try:
|
||||
env = os.environ.copy()
|
||||
env["MEMORY_TENCENTDB_GATEWAY_PORT"] = str(self._port)
|
||||
env["MEMORY_TENCENTDB_GATEWAY_HOST"] = self._host
|
||||
# Note: we deliberately do NOT inject TDAI_GATEWAY_API_KEY into
|
||||
# the child's env from here. Whether the Gateway enforces auth is
|
||||
# the operator's call — they configure it on the Gateway side
|
||||
# (env, yaml, docker run, systemd unit) just like any other
|
||||
# Gateway setting. The supervisor's ``api_key`` is purely the
|
||||
# client-side Bearer token used for outbound requests.
|
||||
|
||||
# Redirect child stdout/stderr to log files instead of PIPE.
|
||||
# Using PIPE without an active reader will deadlock the child once
|
||||
# the pipe buffer (~64 KB) fills up. A log directory next to the
|
||||
# data dir keeps logs inspectable on crash while eliminating the
|
||||
# blocking risk entirely.
|
||||
log_dir = self._resolve_log_dir()
|
||||
try:
|
||||
os.makedirs(log_dir, exist_ok=True)
|
||||
except OSError as e:
|
||||
logger.warning(
|
||||
"memory-tencentdb Gateway: failed to create log dir %s (%s); "
|
||||
"falling back to DEVNULL", log_dir, e,
|
||||
with _startup_singleflight(self._host, self._port):
|
||||
# Another supervisor may have started the Gateway while we were
|
||||
# waiting on the single-flight lock. Re-probe before spawning.
|
||||
if self.is_running():
|
||||
logger.info(
|
||||
"memory-tencentdb Gateway became available at %s while waiting for startup lock",
|
||||
self._base_url,
|
||||
)
|
||||
log_dir = None
|
||||
return True
|
||||
|
||||
if log_dir is not None:
|
||||
stdout_path = os.path.join(log_dir, "gateway.stdout.log")
|
||||
stderr_path = os.path.join(log_dir, "gateway.stderr.log")
|
||||
# Append mode: preserve previous runs for postmortem.
|
||||
self._stdout_log = open(stdout_path, "ab", buffering=0)
|
||||
self._stderr_log = open(stderr_path, "ab", buffering=0)
|
||||
self._stderr_log_path = stderr_path
|
||||
stdout_target: object = self._stdout_log
|
||||
stderr_target: object = self._stderr_log
|
||||
else:
|
||||
stdout_target = subprocess.DEVNULL
|
||||
stderr_target = subprocess.DEVNULL
|
||||
# If we previously spawned a child and it has since died, drop the
|
||||
# stale Popen handle so the new spawn below isn't shadowed by a
|
||||
# zombie reference. Without this, a crashed-then-respawned Gateway
|
||||
# would keep ``self._process`` pointing at the dead PID forever and
|
||||
# ``is_process_alive()`` would mislead the watchdog.
|
||||
self._reap_dead_process()
|
||||
|
||||
self._process = subprocess.Popen(
|
||||
shlex.split(self._gateway_cmd),
|
||||
env=env,
|
||||
stdout=stdout_target,
|
||||
stderr=stderr_target,
|
||||
start_new_session=True, # Detach from parent process group
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error("Failed to start memory-tencentdb Gateway: %s", e)
|
||||
self._close_log_handles()
|
||||
return False
|
||||
# Try to start the Gateway
|
||||
if not self._gateway_cmd:
|
||||
logger.warning(
|
||||
"memory-tencentdb Gateway is not running and no gateway command configured. "
|
||||
"Set MEMORY_TENCENTDB_GATEWAY_CMD environment variable or pass gateway_cmd to supervisor. "
|
||||
"memory-tencentdb memory will be unavailable."
|
||||
)
|
||||
return False
|
||||
|
||||
# Wait for health check
|
||||
return self._wait_for_health()
|
||||
logger.info("Starting memory-tencentdb Gateway: %s", self._gateway_cmd)
|
||||
self._shutdown_requested = False
|
||||
|
||||
try:
|
||||
env = os.environ.copy()
|
||||
# The Python provider historically used MEMORY_TENCENTDB_* while
|
||||
# src/gateway/config.ts reads TDAI_GATEWAY_*. Export both so a
|
||||
# non-default supervisor port cannot accidentally spawn a child
|
||||
# that still binds the default 8420.
|
||||
env["MEMORY_TENCENTDB_GATEWAY_PORT"] = str(self._port)
|
||||
env["MEMORY_TENCENTDB_GATEWAY_HOST"] = self._host
|
||||
env["TDAI_GATEWAY_PORT"] = str(self._port)
|
||||
env["TDAI_GATEWAY_HOST"] = self._host
|
||||
# Note: we deliberately do NOT inject TDAI_GATEWAY_API_KEY into
|
||||
# the child's env from here. Whether the Gateway enforces auth is
|
||||
# the operator's call — they configure it on the Gateway side
|
||||
# (env, yaml, docker run, systemd unit) just like any other
|
||||
# Gateway setting. The supervisor's ``api_key`` is purely the
|
||||
# client-side Bearer token used for outbound requests.
|
||||
|
||||
# Redirect child stdout/stderr to log files instead of PIPE.
|
||||
# Using PIPE without an active reader will deadlock the child once
|
||||
# the pipe buffer (~64 KB) fills up. A log directory next to the
|
||||
# data dir keeps logs inspectable on crash while eliminating the
|
||||
# blocking risk entirely.
|
||||
log_dir = self._resolve_log_dir()
|
||||
try:
|
||||
os.makedirs(log_dir, exist_ok=True)
|
||||
except OSError as e:
|
||||
logger.warning(
|
||||
"memory-tencentdb Gateway: failed to create log dir %s (%s); "
|
||||
"falling back to DEVNULL", log_dir, e,
|
||||
)
|
||||
log_dir = None
|
||||
|
||||
if log_dir is not None:
|
||||
stdout_path = os.path.join(log_dir, "gateway.stdout.log")
|
||||
stderr_path = os.path.join(log_dir, "gateway.stderr.log")
|
||||
# Append mode: preserve previous runs for postmortem.
|
||||
self._stdout_log = open(stdout_path, "ab", buffering=0)
|
||||
self._stderr_log = open(stderr_path, "ab", buffering=0)
|
||||
self._stderr_log_path = stderr_path
|
||||
stdout_target: object = self._stdout_log
|
||||
stderr_target: object = self._stderr_log
|
||||
else:
|
||||
stdout_target = subprocess.DEVNULL
|
||||
stderr_target = subprocess.DEVNULL
|
||||
|
||||
self._process = subprocess.Popen(
|
||||
shlex.split(self._gateway_cmd),
|
||||
env=env,
|
||||
stdout=stdout_target,
|
||||
stderr=stderr_target,
|
||||
start_new_session=True,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error("Failed to start memory-tencentdb Gateway: %s", e)
|
||||
self._close_log_handles()
|
||||
return False
|
||||
|
||||
# Keep the lock until the spawned process becomes healthy; otherwise
|
||||
# a second waiter can observe the port as down during cold start and
|
||||
# launch a duplicate process that immediately hits EADDRINUSE.
|
||||
return self._wait_for_health()
|
||||
|
||||
def _resolve_log_dir(self) -> str:
|
||||
"""Pick a directory to store Gateway stdout/stderr logs.
|
||||
@@ -269,6 +332,10 @@ class GatewaySupervisor:
|
||||
"""Wait for the Gateway to become healthy."""
|
||||
start = time.monotonic()
|
||||
while time.monotonic() - start < HEALTH_CHECK_MAX_WAIT:
|
||||
if self._shutdown_requested:
|
||||
logger.info("memory-tencentdb Gateway startup wait cancelled by shutdown")
|
||||
return False
|
||||
|
||||
# Check if process died
|
||||
if self._process and self._process.poll() is not None:
|
||||
rc = self._process.returncode
|
||||
@@ -303,20 +370,32 @@ class GatewaySupervisor:
|
||||
|
||||
def shutdown(self) -> None:
|
||||
"""Shut down the managed Gateway process (if we started it)."""
|
||||
self._shutdown_requested = True
|
||||
if self._process is None:
|
||||
return
|
||||
|
||||
logger.info("Shutting down memory-tencentdb Gateway...")
|
||||
|
||||
try:
|
||||
# Send SIGTERM for graceful shutdown
|
||||
self._process.terminate()
|
||||
proc = self._process
|
||||
if proc.poll() is None:
|
||||
# The Gateway is started with start_new_session=True. Terminate
|
||||
# the whole process group so `pnpm -> tsx -> node server.ts`
|
||||
# does not leave the real listener orphaned after the top-level
|
||||
# wrapper exits.
|
||||
try:
|
||||
os.killpg(os.getpgid(proc.pid), signal.SIGTERM)
|
||||
except Exception:
|
||||
proc.terminate()
|
||||
try:
|
||||
self._process.wait(timeout=10)
|
||||
proc.wait(timeout=10)
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.warning("memory-tencentdb Gateway did not exit in 10s, sending SIGKILL")
|
||||
self._process.kill()
|
||||
self._process.wait(timeout=5)
|
||||
try:
|
||||
os.killpg(os.getpgid(proc.pid), signal.SIGKILL)
|
||||
except Exception:
|
||||
proc.kill()
|
||||
proc.wait(timeout=5)
|
||||
except Exception as e:
|
||||
logger.warning("Error shutting down memory-tencentdb Gateway: %s", e)
|
||||
finally:
|
||||
|
||||
@@ -231,6 +231,69 @@ def test_reap_dead_process_keeps_alive_handle():
|
||||
assert sup._process is alive
|
||||
|
||||
|
||||
def test_ensure_running_singleflight_prevents_duplicate_spawn(monkeypatch):
|
||||
"""Concurrent supervisors for one port must spawn at most one Gateway."""
|
||||
port = 28421
|
||||
running = False
|
||||
spawn_envs = []
|
||||
spawn_lock = threading.Lock()
|
||||
|
||||
class FakePopen:
|
||||
pid = 12345
|
||||
returncode = None
|
||||
|
||||
def poll(self):
|
||||
return None
|
||||
|
||||
def fake_is_running(self):
|
||||
return running
|
||||
|
||||
def fake_wait_for_health(self):
|
||||
nonlocal running
|
||||
time.sleep(0.1)
|
||||
running = True
|
||||
return True
|
||||
|
||||
def fake_popen(_argv, *, env, stdout, stderr, start_new_session):
|
||||
with spawn_lock:
|
||||
spawn_envs.append(dict(env))
|
||||
assert start_new_session is True
|
||||
return FakePopen()
|
||||
|
||||
monkeypatch.setattr(supervisor_module.GatewaySupervisor, "is_running", fake_is_running)
|
||||
monkeypatch.setattr(supervisor_module.GatewaySupervisor, "_wait_for_health", fake_wait_for_health)
|
||||
monkeypatch.setattr(supervisor_module.subprocess, "Popen", fake_popen)
|
||||
|
||||
supervisors = [
|
||||
supervisor_module.GatewaySupervisor(
|
||||
host="127.0.0.1",
|
||||
port=port,
|
||||
gateway_cmd="fake gateway",
|
||||
)
|
||||
for _ in range(6)
|
||||
]
|
||||
barrier = threading.Barrier(len(supervisors))
|
||||
results = []
|
||||
|
||||
def worker(sup):
|
||||
barrier.wait()
|
||||
results.append(sup.ensure_running())
|
||||
|
||||
threads = [threading.Thread(target=worker, args=(sup,)) for sup in supervisors]
|
||||
for t in threads:
|
||||
t.start()
|
||||
for t in threads:
|
||||
t.join(timeout=3)
|
||||
|
||||
assert results == [True] * len(supervisors)
|
||||
assert len(spawn_envs) == 1
|
||||
env = spawn_envs[0]
|
||||
assert env["MEMORY_TENCENTDB_GATEWAY_PORT"] == str(port)
|
||||
assert env["MEMORY_TENCENTDB_GATEWAY_HOST"] == "127.0.0.1"
|
||||
assert env["TDAI_GATEWAY_PORT"] == str(port)
|
||||
assert env["TDAI_GATEWAY_HOST"] == "127.0.0.1"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Watchdog: detects death, resurrects, and reattaches
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -22,6 +22,7 @@ import path from "node:path";
|
||||
import fs from "node:fs";
|
||||
import { createRequire } from "node:module";
|
||||
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/core";
|
||||
import registerClientOpenClawPlugin from "./openclaw-plugin/index.js";
|
||||
import { parseConfig } from "./src/config.js";
|
||||
import type { MemoryTdaiConfig } from "./src/config.js";
|
||||
import { executeReadCos, READ_COS_TOOL_SCHEMA, READ_COS_TOOL_NAME, READ_COS_TOOL_DESCRIPTION } from "./src/core/tools/read-cos.js";
|
||||
@@ -29,6 +30,7 @@ import { LocalStorageBackend } from "./src/core/storage/local-backend.js";
|
||||
import { StaticCredentialProvider } from "./src/core/storage/credential-provider.js";
|
||||
import type { IStorageBackend } from "./src/core/storage/types.js";
|
||||
import { registerOffload } from "./src/offload/index.js";
|
||||
import { registerOffloadClient } from "./src/offload-client/index.js";
|
||||
import {
|
||||
setPreferredEmbeddedAgentRuntime,
|
||||
prewarmEmbeddedAgent,
|
||||
@@ -52,6 +54,22 @@ import { resolveOpenClawStateDir } from "./src/utils/openclaw-state-dir.js";
|
||||
|
||||
const TAG = "[memory-tdai]";
|
||||
|
||||
type OpenClawAdapterMode = "local" | "client";
|
||||
|
||||
function resolveOpenClawAdapterMode(rawPluginConfig: Record<string, unknown> | undefined): OpenClawAdapterMode {
|
||||
const rawMode = typeof rawPluginConfig?.mode === "string"
|
||||
? rawPluginConfig.mode.trim().toLowerCase()
|
||||
: "";
|
||||
|
||||
if (rawMode === "client" || rawMode === "gateway" || rawMode === "remote") {
|
||||
return "client";
|
||||
}
|
||||
|
||||
// Default to local/function mode: OpenClaw calls this plugin in-process and
|
||||
// memory processing uses the host LLM runner (no standalone Gateway needed).
|
||||
return "local";
|
||||
}
|
||||
|
||||
/**
|
||||
* Epoch ms when the plugin was registered (cold-start timestamp).
|
||||
* Used as a fallback cursor in performAutoCapture when no checkpoint
|
||||
@@ -158,6 +176,13 @@ export default function register(api: OpenClawPluginApi) {
|
||||
}
|
||||
|
||||
// ─── Full / discovery mode: complete runtime initialization ───
|
||||
const rawPluginConfigForMode = api.pluginConfig as Record<string, unknown> | undefined;
|
||||
const adapterMode = resolveOpenClawAdapterMode(rawPluginConfigForMode);
|
||||
if (adapterMode === "client") {
|
||||
api.logger.info?.(`${TAG} mode=client: delegating to memory-tencentdb-client adapter`);
|
||||
return registerClientOpenClawPlugin(api as any);
|
||||
}
|
||||
|
||||
pluginStartTimestamp = Date.now();
|
||||
setPreferredEmbeddedAgentRuntime(api.runtime.agent);
|
||||
// Reset reporter singleton so config changes take effect on hot-reload.
|
||||
@@ -189,7 +214,7 @@ export default function register(api: OpenClawPluginApi) {
|
||||
`pipeline=(everyN=${cfg.pipeline.everyNConversations}, warmup=${cfg.pipeline.enableWarmup}, l1Idle=${cfg.pipeline.l1IdleTimeoutSeconds}s, l2DelayAfterL1=${cfg.pipeline.l2DelayAfterL1Seconds}s, l2Min=${cfg.pipeline.l2MinIntervalSeconds}s, l2Max=${cfg.pipeline.l2MaxIntervalSeconds}s, activeWindow=${cfg.pipeline.sessionActiveWindowHours}h), ` +
|
||||
`persona(triggerEvery=${cfg.persona.triggerEveryN}, backupCount=${cfg.persona.backupCount}, sceneBackupCount=${cfg.persona.sceneBackupCount}), ` +
|
||||
`memoryCleanup(enabled=${cfg.memoryCleanup.enabled}, retentionDays=${cfg.memoryCleanup.retentionDays ?? "(disabled)"}, cleanTime=${cfg.memoryCleanup.cleanTime}), ` +
|
||||
`offload(enabled=${cfg.offload.enabled}, backendUrl=${cfg.offload.backendUrl ?? "(none)"}, mildRatio=${cfg.offload.mildOffloadRatio}, aggressiveRatio=${cfg.offload.aggressiveCompressRatio}, retentionDays=${cfg.offload.offloadRetentionDays})`,
|
||||
`offload(enabled=${cfg.offload.enabled}, mode=${cfg.offload.mode}, ${cfg.offload.mode === "client" ? `serverUrl=${cfg.offload.serverUrl ?? "(none)"}` : `backendUrl=${cfg.offload.backendUrl ?? "(none)"}, mildRatio=${cfg.offload.mildOffloadRatio}, aggressiveRatio=${cfg.offload.aggressiveCompressRatio}`}, retentionDays=${cfg.offload.offloadRetentionDays})`,
|
||||
);
|
||||
} catch (err) {
|
||||
api.logger.error(`${TAG} Config parsing failed: ${err instanceof Error ? err.message : String(err)}`);
|
||||
@@ -232,7 +257,7 @@ export default function register(api: OpenClawPluginApi) {
|
||||
try {
|
||||
ensurePluginHookPolicy({
|
||||
rootConfig: api.config,
|
||||
runtimeConfig: api.runtime?.config,
|
||||
runtimeConfig: (api.runtime as any)?.config,
|
||||
logger: api.logger,
|
||||
});
|
||||
} catch (err) {
|
||||
@@ -556,7 +581,10 @@ export default function register(api: OpenClawPluginApi) {
|
||||
storagePromise = (async (): Promise<IStorageBackend> => {
|
||||
if (cosSecretId && cosSecretKey && cosBucket) {
|
||||
try {
|
||||
const { CosStorageBackend } = await import("./src/integrations/cos/cos-backend.js");
|
||||
const cosBackendModulePath = ["./src/integrations/cos", "cos-backend.js"].join("/");
|
||||
const { CosStorageBackend } = await import(cosBackendModulePath) as {
|
||||
CosStorageBackend: new (options: any) => IStorageBackend;
|
||||
};
|
||||
const backend = new CosStorageBackend({
|
||||
credentialProvider: new StaticCredentialProvider({
|
||||
secretId: cosSecretId,
|
||||
@@ -954,8 +982,24 @@ export default function register(api: OpenClawPluginApi) {
|
||||
if (cfg.offload.enabled) {
|
||||
api.logger.debug?.(`${TAG} Offload enabled, registering offload module...`);
|
||||
try {
|
||||
registerOffload(api, cfg.offload);
|
||||
api.logger.debug?.(`${TAG} Offload module registered successfully`);
|
||||
if (cfg.offload.mode === "client") {
|
||||
// New: stateless client mode — all compression delegated to server
|
||||
registerOffloadClient(api as any, {
|
||||
enabled: cfg.offload.enabled,
|
||||
serverUrl: cfg.offload.serverUrl ?? "",
|
||||
apiKey: cfg.offload.apiKey ?? "",
|
||||
serviceId: cfg.offload.serviceId ?? "",
|
||||
agentName: cfg.offload.agentName ?? "default",
|
||||
compactionRatio: cfg.offload.compactionRatio ?? 0.5,
|
||||
ingestTimeoutMs: cfg.offload.ingestTimeoutMs ?? 5000,
|
||||
compactionTimeoutMs: cfg.offload.compactionTimeoutMs ?? 30000,
|
||||
});
|
||||
api.logger.debug?.(`${TAG} Offload client module registered (mode=client)`);
|
||||
} else {
|
||||
// Legacy: full local L3 compression
|
||||
registerOffload(api, cfg.offload);
|
||||
api.logger.debug?.(`${TAG} Offload module registered successfully (mode=${cfg.offload.mode})`);
|
||||
}
|
||||
} catch (err) {
|
||||
api.logger.error(`${TAG} Offload module registration failed: ${err instanceof Error ? err.message : String(err)}`);
|
||||
}
|
||||
|
||||
@@ -2,7 +2,8 @@
|
||||
* memory-tencentdb-client — OpenClaw 记忆插件(客户端接入版)
|
||||
*
|
||||
* 通过 @tencentdb-agent-memory/memory-sdk-ts 连接远端 memory server,
|
||||
* 提供四层记忆的自动捕获、召回和工具调用能力。
|
||||
* 提供四层记忆的自动捕获、召回和工具调用能力,
|
||||
* 并集成 offload 上下文压缩/摘要服务。
|
||||
*
|
||||
* 本插件不包含任何数据处理逻辑(无 VDB/Embedding/Pipeline),
|
||||
* 所有操作委托给远端 server。
|
||||
@@ -14,6 +15,8 @@ import { performCapture } from "./src/hooks/capture.js";
|
||||
import { handleMemorySearch } from "./src/tools/memory-search.js";
|
||||
import { handleConversationSearch } from "./src/tools/conversation-search.js";
|
||||
import { handleReadCos } from "./src/tools/read-cos.js";
|
||||
// @ts-ignore — rootDir mismatch in openclaw-plugin tsconfig; resolved at bundle time by tsdown
|
||||
import { registerOffloadClient } from "../src/offload-client/index.js";
|
||||
|
||||
const TAG = "[memory-client]";
|
||||
|
||||
@@ -22,6 +25,7 @@ interface ServerConfig {
|
||||
url?: string;
|
||||
apiKey?: string;
|
||||
instanceId?: string;
|
||||
rejectUnauthorized?: boolean;
|
||||
}
|
||||
interface RecallConfig {
|
||||
maxResults?: number;
|
||||
@@ -31,10 +35,18 @@ interface RecallConfig {
|
||||
interface CaptureConfig {
|
||||
enabled?: boolean;
|
||||
}
|
||||
interface OffloadConfig {
|
||||
enabled?: boolean;
|
||||
serverUrl?: string;
|
||||
compactionRatio?: number;
|
||||
ingestTimeoutMs?: number;
|
||||
compactionTimeoutMs?: number;
|
||||
}
|
||||
interface PluginConfig {
|
||||
server?: ServerConfig;
|
||||
recall?: RecallConfig;
|
||||
capture?: CaptureConfig;
|
||||
offload?: OffloadConfig;
|
||||
}
|
||||
|
||||
// Matches OpenClaw plugin register() signature: export default function register(api)
|
||||
@@ -44,6 +56,7 @@ export default function register(api: any) {
|
||||
const server = cfg.server ?? {};
|
||||
const recall = cfg.recall ?? {};
|
||||
const capture = cfg.capture ?? {};
|
||||
const offload = cfg.offload ?? {};
|
||||
|
||||
const serverUrl = server.url || "http://127.0.0.1:8420";
|
||||
const apiKey = server.apiKey || "sk-xxxx";
|
||||
@@ -52,6 +65,13 @@ export default function register(api: any) {
|
||||
const includePersona = recall.includePersona !== false;
|
||||
const includeSceneNav = recall.includeSceneNav !== false;
|
||||
const captureEnabled = capture.enabled !== false;
|
||||
const rejectUnauthorized = server.rejectUnauthorized !== false;
|
||||
|
||||
// Offload config (auto-inherits from server config)
|
||||
const offloadEnabled = offload.enabled ?? false;
|
||||
const offloadServerUrl = offload.serverUrl || serverUrl;
|
||||
const offloadCompactionRatio = offload.compactionRatio ?? 0.5;
|
||||
const offloadCompactionTimeoutMs = offload.compactionTimeoutMs ?? 30000;
|
||||
|
||||
// ── Initialize SDK ──
|
||||
// NOTE: pass config (not a raw Transport) so client.readFile can lazily
|
||||
@@ -60,12 +80,15 @@ export default function register(api: any) {
|
||||
endpoint: serverUrl,
|
||||
apiKey,
|
||||
serviceId: instanceId,
|
||||
rejectUnauthorized,
|
||||
});
|
||||
|
||||
|
||||
|
||||
api.logger.info?.(
|
||||
`${TAG} Initialized: server=${serverUrl}, instance=${instanceId}, ` +
|
||||
`recall(persona=${includePersona},sceneNav=${includeSceneNav},max=${recallMaxResults}), ` +
|
||||
`capture=${captureEnabled}`,
|
||||
`capture=${captureEnabled}, offload=${offloadEnabled}, rejectUnauthorized=${rejectUnauthorized}`,
|
||||
);
|
||||
|
||||
// ── Register Tools (same pattern as extensions/memory-tencentdb/index.ts) ──
|
||||
@@ -249,4 +272,17 @@ export default function register(api: any) {
|
||||
} else {
|
||||
api.logger.info?.(`${TAG} capture disabled by config`);
|
||||
}
|
||||
|
||||
// ── Offload: unified registration via registerOffloadClient ────────────────────
|
||||
if (offloadEnabled) {
|
||||
registerOffloadClient(api, {
|
||||
enabled: true,
|
||||
serverUrl: offloadServerUrl,
|
||||
apiKey,
|
||||
serviceId: instanceId,
|
||||
agentName: "default",
|
||||
compactionRatio: offloadCompactionRatio,
|
||||
compactionTimeoutMs: offloadCompactionTimeoutMs,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -35,6 +35,11 @@
|
||||
"type": "string",
|
||||
"default": "default",
|
||||
"description": "Memory instance id(HTTP header x-tdai-service-id)"
|
||||
},
|
||||
"rejectUnauthorized": {
|
||||
"type": "boolean",
|
||||
"default": true,
|
||||
"description": "是否校验 HTTPS 证书。默认 true;仅自签证书测试环境需要显式设为 false"
|
||||
}
|
||||
}
|
||||
},
|
||||
@@ -69,6 +74,36 @@
|
||||
"description": "是否启用自动对话捕获 (L0)"
|
||||
}
|
||||
}
|
||||
},
|
||||
"offload": {
|
||||
"type": "object",
|
||||
"description": "Offload 压缩/摘要服务配置(可选,不配则复用 server 配置)",
|
||||
"properties": {
|
||||
"enabled": {
|
||||
"type": "boolean",
|
||||
"default": false,
|
||||
"description": "是否启用 Offload 上下文压缩"
|
||||
},
|
||||
"serverUrl": {
|
||||
"type": "string",
|
||||
"description": "Offload server URL。不填则自动从 server.url 推导(替换端口为 9100)"
|
||||
},
|
||||
"compactionRatio": {
|
||||
"type": "number",
|
||||
"default": 0.5,
|
||||
"description": "上下文占比超过此阈值时触发压缩(0-1)"
|
||||
},
|
||||
"ingestTimeoutMs": {
|
||||
"type": "number",
|
||||
"default": 5000,
|
||||
"description": "Ingest 请求超时(毫秒)"
|
||||
},
|
||||
"compactionTimeoutMs": {
|
||||
"type": "number",
|
||||
"default": 30000,
|
||||
"description": "Compaction 请求超时(毫秒)"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+33
-14
@@ -7,17 +7,32 @@
|
||||
"onStartup": true
|
||||
},
|
||||
"contracts": {
|
||||
"tools": ["tdai_memory_search", "tdai_conversation_search"]
|
||||
"tools": ["tdai_memory_search", "tdai_conversation_search", "tdai_read_cos"]
|
||||
},
|
||||
"configSchema": {
|
||||
"type": "object",
|
||||
"additionalProperties": true,
|
||||
"properties": {
|
||||
"mode": {
|
||||
"type": "string",
|
||||
"enum": ["local", "function", "client", "gateway", "remote"],
|
||||
"default": "local",
|
||||
"description": "OpenClaw 接入模式:local/function(默认,本地进程内运行,使用 OpenClaw function/host LLM runner)或 client/gateway/remote(轻量客户端模式,连接外部 Memory Gateway)"
|
||||
},
|
||||
"storeBackend": {
|
||||
"type": "string",
|
||||
"enum": ["sqlite", "tcvdb"],
|
||||
"default": "sqlite",
|
||||
"description": "存储后端:sqlite(本地 SQLite + sqlite-vec)或 tcvdb(腾讯云向量数据库)"
|
||||
"description": "存储后端:sqlite(本地 SQLite + sqlite-vec)或 tcvdb(腾讯云向量数据库)。仅 local/function 模式生效"
|
||||
},
|
||||
"server": {
|
||||
"type": "object",
|
||||
"description": "client/gateway/remote 模式下的外部 Memory Gateway 连接配置",
|
||||
"properties": {
|
||||
"url": { "type": "string", "default": "http://127.0.0.1:8420", "description": "Memory Gateway URL" },
|
||||
"apiKey": { "type": "string", "default": "local", "description": "API Key for Gateway authentication" },
|
||||
"instanceId": { "type": "string", "default": "default", "description": "Memory instance id(HTTP header x-tdai-service-id)" }
|
||||
}
|
||||
},
|
||||
"capture": {
|
||||
"type": "object",
|
||||
@@ -140,23 +155,27 @@
|
||||
},
|
||||
"offload": {
|
||||
"type": "object",
|
||||
"description": "Context Offload 设置 — 多层上下文压缩系统(独立开关,默认关闭)",
|
||||
"description": "Context Offload 设置 — 多层上下文压缩系统(独立开关,默认关闭)。支持 local(本地 LLM 处理)和 client(复用 server.url 远端服务)两种模式",
|
||||
"properties": {
|
||||
"enabled": { "type": "boolean", "default": false, "description": "是否启用 Context Offload(默认关闭,不影响 Memory 功能)" },
|
||||
"model": { "type": "string", "description": "Offload 使用的 LLM 模型(格式: provider/model),未填写时使用 openclaw 默认模型" },
|
||||
"temperature": { "type": "number", "default": 0.2, "description": "LLM 温度参数" },
|
||||
"forceTriggerThreshold": { "type": "number", "default": 4, "description": "累积多少个 tool pair 后强制触发 L1" },
|
||||
"dataDir": { "type": "string", "description": "自定义数据目录(绝对路径),默认 ~/.openclaw/context-offload" },
|
||||
"mode": { "type": "string", "enum": ["local", "backend", "client", "collect"], "description": "Offload 运行模式:local(本地 LLM 处理)、backend(走 backendUrl 远端 v1 服务)、client(复用 server.url 远端 offload v2 服务)、collect(仅数据采集,不压缩)。未设置时自动推导:有 backendUrl → backend,有 server.url → client,否则 → local" },
|
||||
"model": { "type": "string", "description": "【local 模式】Offload 使用的 LLM 模型(格式: provider/model),未填写时使用 openclaw 默认模型" },
|
||||
"temperature": { "type": "number", "default": 0.2, "description": "【local 模式】LLM 温度参数" },
|
||||
"forceTriggerThreshold": { "type": "number", "default": 4, "description": "【local 模式】累积多少个 tool pair 后强制触发 L1" },
|
||||
"dataDir": { "type": "string", "description": "【local 模式】自定义数据目录(绝对路径),默认 ~/.openclaw/context-offload" },
|
||||
"defaultContextWindow": { "type": "number", "default": 200000, "description": "默认上下文窗口大小" },
|
||||
"maxPairsPerBatch": { "type": "number", "default": 20, "description": "L1 每批最大 tool pair 数" },
|
||||
"l2NullThreshold": { "type": "number", "default": 4, "description": "offload.jsonl 中 node_id=null 数量达到此阈值时触发 L2" },
|
||||
"l2TimeoutSeconds": { "type": "number", "default": 300, "description": "距上次 L2 超过此秒数时触发 L2" },
|
||||
"maxPairsPerBatch": { "type": "number", "default": 20, "description": "【local 模式】L1 每批最大 tool pair 数" },
|
||||
"l2NullThreshold": { "type": "number", "default": 4, "description": "【local 模式】offload.jsonl 中 node_id=null 数量达到此阈值时触发 L2" },
|
||||
"l2TimeoutSeconds": { "type": "number", "default": 300, "description": "【local 模式】距上次 L2 超过此秒数时触发 L2" },
|
||||
"mildOffloadRatio": { "type": "number", "default": 0.5, "description": "温和压缩触发比例(占 context window)" },
|
||||
"aggressiveCompressRatio": { "type": "number", "default": 0.85, "description": "激进压缩触发比例" },
|
||||
"mmdMaxTokenRatio": { "type": "number", "default": 0.2, "description": "MMD 注入 token 预算比例" },
|
||||
"backendUrl": { "type": "string", "description": "后端服务 URL(如 https://offload-api.example.com),配置后 L1/L1.5/L2/L4 走后端" },
|
||||
"backendApiKey": { "type": "string", "description": "后端 API 认证 token" },
|
||||
"backendTimeoutMs": { "type": "number", "default": 10000, "description": "后端调用超时(毫秒)" }
|
||||
"mmdMaxTokenRatio": { "type": "number", "default": 0.2, "description": "【local 模式】MMD 注入 token 预算比例" },
|
||||
"compactionRatio": { "type": "number", "default": 0.5, "description": "【client 模式】上下文占比超过此阈值时触发压缩(0-1),复用 server.url 作为后端地址" },
|
||||
"ingestTimeoutMs": { "type": "number", "default": 5000, "description": "【client 模式】Ingest 请求超时(毫秒)" },
|
||||
"compactionTimeoutMs": { "type": "number", "default": 30000, "description": "【client 模式】Compaction 请求超时(毫秒)" },
|
||||
"backendUrl": { "type": "string", "description": "【已废弃,请使用 mode=client】后端服务 URL,配置后 L1/L1.5/L2/L4 走后端" },
|
||||
"backendApiKey": { "type": "string", "description": "【已废弃,请使用 mode=client】后端 API 认证 token" },
|
||||
"backendTimeoutMs": { "type": "number", "default": 10000, "description": "【已废弃,请使用 mode=client】后端调用超时(毫秒)" }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+6
-2
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@tencentdb-agent-memory/memory-tencentdb",
|
||||
"version": "1.0.0-beta.1",
|
||||
"version": "1.0.0",
|
||||
"description": "Four-layer local memory system plugin for OpenClaw — auto-captures, structures, and profiles conversational knowledge using local LLM + SQLite vector search (L0→L1→L2→L3 pipeline)",
|
||||
"type": "module",
|
||||
"main": "./dist/index.mjs",
|
||||
@@ -45,11 +45,14 @@
|
||||
"scripts/read-local-memory/dist/",
|
||||
"scripts/memory-tencentdb-ctl.sh",
|
||||
"scripts/install_hermes_memory_tencentdb.sh",
|
||||
"scripts/install-hermes-plugin-v2.sh",
|
||||
"scripts/install-openclaw-plugin-v2.sh",
|
||||
"scripts/README.memory-tencentdb-ctl.md",
|
||||
"src",
|
||||
"scripts/openclaw-after-tool-call-messages.patch.sh",
|
||||
"scripts/setup-offload.sh",
|
||||
"hermes-plugin/",
|
||||
"openclaw-plugin/",
|
||||
"openclaw.plugin.json",
|
||||
"README.md",
|
||||
"CHANGELOG.md",
|
||||
@@ -90,6 +93,7 @@
|
||||
"@opentelemetry/sdk-node": "^0.218.0",
|
||||
"@opentelemetry/sdk-trace-base": "^2.7.1",
|
||||
"@opentelemetry/semantic-conventions": "^1.41.1",
|
||||
"@tencentdb-agent-memory/memory-sdk-ts": "^1.0.0",
|
||||
"@tencentdb-agent-memory/tcvdb-text": "^0.1.1",
|
||||
"ai": "^6.0.164",
|
||||
"crc-32": "^1.2.2",
|
||||
@@ -110,7 +114,7 @@
|
||||
"cos-nodejs-sdk-v5": "^2.15.4",
|
||||
"ioredis": "^5.10.1",
|
||||
"kafkajs": "^2.2.4",
|
||||
"opik": "^1.0.0"
|
||||
"opik": "^1.11.14"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"node-llama-cpp": "^3.16.2",
|
||||
|
||||
@@ -121,9 +121,7 @@ if (fs.existsSync(file)) {
|
||||
}
|
||||
|
||||
config.plugins ??= {};
|
||||
config.plugins.slots ??= {};
|
||||
config.plugins.entries ??= {};
|
||||
config.plugins.slots.memory = pluginId;
|
||||
|
||||
const entry = config.plugins.entries[pluginId] ?? {};
|
||||
entry.enabled = true;
|
||||
@@ -173,7 +171,6 @@ cat >&2 <<EOF
|
||||
|
||||
[install-openclaw-plugin-v2] Done.
|
||||
Memory plugin configured:
|
||||
plugins.slots.memory = "$MEMORY_PLUGIN_ID"
|
||||
plugins.entries["$MEMORY_PLUGIN_ID"].enabled = true
|
||||
$HOOK_SUMMARY
|
||||
server.url = "$TDAI_MEMORY_ENDPOINT"
|
||||
|
||||
Executable
+162
@@ -0,0 +1,162 @@
|
||||
#!/usr/bin/env bash
|
||||
# ═══════════════════════════════════════════════════════════════════
|
||||
# start-offload-local.sh — 本地启动 Offload Server(零外部依赖)
|
||||
# ═══════════════════════════════════════════════════════════════════
|
||||
#
|
||||
# 使用 LocalStorageBackend(本地文件) + LocalStateBackend(进程内存)
|
||||
# 替代生产环境的 COS + Redis,适合本地开发调试。
|
||||
#
|
||||
# 用法:
|
||||
# bash scripts/start-offload-local.sh [start|stop|restart|status]
|
||||
# 默认: start(后台运行)
|
||||
#
|
||||
# 环境变量(可选):
|
||||
# PORT — 服务端口,默认 9100
|
||||
# DATA_DIR — 数据存储目录,默认 /tmp/openclaw/data
|
||||
# LOG_DIR — 日志输出目录,默认 /tmp/openclaw/logs
|
||||
# API_KEY — 认证 API Key,默认 "dev-local-key"
|
||||
# LLM_BASE_URL — LLM API 地址(L1/L1.5/L2 需要),默认空
|
||||
# LLM_API_KEY — LLM API Key,默认空
|
||||
# OPIK_ENABLED — 是否启用 Opik 追踪,默认 "false"
|
||||
# OPIK_URL_OVERRIDE — Opik Server 地址
|
||||
# OPIK_API_KEY — Opik API Key
|
||||
# OPIK_WORKSPACE — Opik Workspace,默认 "default"
|
||||
# OPIK_PROJECT_NAME — Opik Project,默认 "openclaw-offload-server"
|
||||
# L2_NULL_THRESHOLD — L2 触发阈值,默认 6(设为 999999 可禁用 L2)
|
||||
# AGGRESSIVE_COMPRESS_RATIO — aggressive 压缩触发比例,默认 0.85(设为 999 可禁用)
|
||||
# EMERGENCY_COMPRESS_RATIO — emergency 压缩触发比例,默认 0.95(设为 999 可禁用)
|
||||
# ═══════════════════════════════════════════════════════════════════
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
PROJECT_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||
|
||||
# ── 配置 ──
|
||||
PORT="${PORT:-9100}"
|
||||
DATA_DIR="${DATA_DIR:-/tmp/openclaw/data}"
|
||||
LOG_DIR="${LOG_DIR:-/tmp/openclaw/logs}"
|
||||
API_KEY="${API_KEY:-dev-local-key}"
|
||||
LLM_BASE_URL="${LLM_BASE_URL:-https://tokenhub.tencentmaas.com/v1}"
|
||||
LLM_API_KEY="${LLM_API_KEY:-sk-yYBudowP30M9t4X99vBOtNzhqz4a39jud4i3Br328mh87qcr}"
|
||||
LLM_MODEL="${LLM_MODEL:-ep-91sgogyz}"
|
||||
OPIK_ENABLED="${OPIK_ENABLED:-true}"
|
||||
OPIK_URL_OVERRIDE="${OPIK_URL_OVERRIDE:-http://21.6.226.49:5173/api}"
|
||||
OPIK_API_KEY="${OPIK_API_KEY:-}"
|
||||
OPIK_WORKSPACE="${OPIK_WORKSPACE:-default}"
|
||||
OPIK_PROJECT_NAME="${OPIK_PROJECT_NAME:-offload_server_49}"
|
||||
L2_NULL_THRESHOLD="${L2_NULL_THRESHOLD:-6}"
|
||||
AGGRESSIVE_COMPRESS_RATIO="${AGGRESSIVE_COMPRESS_RATIO:-0.85}"
|
||||
EMERGENCY_COMPRESS_RATIO="${EMERGENCY_COMPRESS_RATIO:-0.95}"
|
||||
PID_FILE="/tmp/openclaw/offload.pid"
|
||||
|
||||
# ── 子命令 ──
|
||||
CMD="${1:-start}"
|
||||
|
||||
do_stop() {
|
||||
if [ -f "${PID_FILE}" ]; then
|
||||
local pid
|
||||
pid=$(cat "${PID_FILE}")
|
||||
if kill -0 "${pid}" 2>/dev/null; then
|
||||
kill "${pid}"
|
||||
echo "[OK] Stopped (pid=${pid})"
|
||||
else
|
||||
echo "[WARN] Process ${pid} not running"
|
||||
fi
|
||||
rm -f "${PID_FILE}"
|
||||
else
|
||||
echo "[WARN] No PID file found"
|
||||
fi
|
||||
}
|
||||
|
||||
do_status() {
|
||||
if [ -f "${PID_FILE}" ] && kill -0 "$(cat "${PID_FILE}")" 2>/dev/null; then
|
||||
echo "[OK] Running (pid=$(cat "${PID_FILE}"), port=${PORT})"
|
||||
else
|
||||
echo "[OFF] Not running"
|
||||
fi
|
||||
}
|
||||
|
||||
do_start() {
|
||||
# Kill existing if running
|
||||
if [ -f "${PID_FILE}" ] && kill -0 "$(cat "${PID_FILE}")" 2>/dev/null; then
|
||||
kill "$(cat "${PID_FILE}")" 2>/dev/null
|
||||
sleep 1
|
||||
fi
|
||||
|
||||
mkdir -p "${DATA_DIR}" "${LOG_DIR}"
|
||||
|
||||
local LOG_FILE="${LOG_DIR}/offload-server-$(date +%Y%m%d).log"
|
||||
|
||||
echo "═══════════════════════════════════════════════════════════"
|
||||
echo " Offload Server — Local Mode (zero dependencies)"
|
||||
echo "═══════════════════════════════════════════════════════════"
|
||||
echo " Port: ${PORT}"
|
||||
echo " Data: ${DATA_DIR}"
|
||||
echo " Log: ${LOG_FILE}"
|
||||
echo " API Key: ${API_KEY:0:8}..."
|
||||
echo " LLM URL: ${LLM_BASE_URL}"
|
||||
echo " LLM Model: ${LLM_MODEL}"
|
||||
echo " Opik: ${OPIK_ENABLED} (${OPIK_URL_OVERRIDE:-disabled})"
|
||||
echo " L2 threshold: ${L2_NULL_THRESHOLD}"
|
||||
echo " Aggressive ratio: ${AGGRESSIVE_COMPRESS_RATIO}"
|
||||
echo " Emergency ratio: ${EMERGENCY_COMPRESS_RATIO}"
|
||||
echo "═══════════════════════════════════════════════════════════"
|
||||
|
||||
cd "${PROJECT_ROOT}"
|
||||
|
||||
# 动态生成临时配置文件,将 standalone yaml 与 offload 覆盖项合并
|
||||
local OVERRIDE_CONFIG="/tmp/openclaw/tdai-gateway.override.yaml"
|
||||
mkdir -p /tmp/openclaw
|
||||
cat "${PROJECT_ROOT}/tdai-gateway.standalone.yaml" > "${OVERRIDE_CONFIG}"
|
||||
cat >> "${OVERRIDE_CONFIG}" << YAML_EOF
|
||||
|
||||
# ── 动态注入的 offload 覆盖配置(由 start-offload-local.sh 生成)──
|
||||
offload:
|
||||
l2NullThreshold: ${L2_NULL_THRESHOLD}
|
||||
aggressiveCompressRatio: ${AGGRESSIVE_COMPRESS_RATIO}
|
||||
emergencyCompressRatio: ${EMERGENCY_COMPRESS_RATIO}
|
||||
YAML_EOF
|
||||
|
||||
TDAI_GATEWAY_CONFIG="${OVERRIDE_CONFIG}" \
|
||||
TDAI_GATEWAY_PORT="${PORT}" \
|
||||
TDAI_GATEWAY_HOST="0.0.0.0" \
|
||||
TDAI_DATA_DIR="${DATA_DIR}" \
|
||||
STATE_BACKEND="local" \
|
||||
TDAI_V2_API_KEY="${API_KEY}" \
|
||||
TDAI_LLM_API_KEY="${LLM_API_KEY}" \
|
||||
TDAI_LLM_BASE_URL="${LLM_BASE_URL}" \
|
||||
TDAI_LLM_MODEL="${LLM_MODEL}" \
|
||||
OPIK_ENABLED="${OPIK_ENABLED}" \
|
||||
OPIK_URL_OVERRIDE="${OPIK_URL_OVERRIDE}" \
|
||||
OPIK_API_KEY="${OPIK_API_KEY}" \
|
||||
OPIK_WORKSPACE="${OPIK_WORKSPACE}" \
|
||||
OPIK_PROJECT_NAME="${OPIK_PROJECT_NAME}" \
|
||||
MEMORY_MAX_BODY_BYTES="10485760" \
|
||||
nohup npx tsx src/gateway/server.ts >> "${LOG_FILE}" 2>&1 &
|
||||
|
||||
echo $! > "${PID_FILE}"
|
||||
sleep 2
|
||||
|
||||
if kill -0 "$(cat "${PID_FILE}")" 2>/dev/null; then
|
||||
echo "[OK] Started (pid=$(cat "${PID_FILE}"), log=${LOG_FILE})"
|
||||
echo ""
|
||||
echo " 验证: curl -s http://127.0.0.1:${PORT}/health"
|
||||
echo " 停止: bash scripts/start-offload-local.sh stop"
|
||||
echo " 日志: tail -f ${LOG_FILE}"
|
||||
else
|
||||
echo "[FAIL] Process exited, check log: ${LOG_FILE}"
|
||||
tail -10 "${LOG_FILE}"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
case "${CMD}" in
|
||||
start) do_start ;;
|
||||
stop) do_stop ;;
|
||||
restart) do_stop; sleep 1; do_start ;;
|
||||
status) do_status ;;
|
||||
*)
|
||||
echo "用法: bash scripts/start-offload-local.sh [start|stop|restart|status]"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
@@ -63,6 +63,23 @@ print(core["content"])
|
||||
# L3: write core memory
|
||||
client.write_core("# User Profile\n...")
|
||||
|
||||
# Offload v2: send tool pairs for server-side L1 async processing (fire-and-forget)
|
||||
client.offload_ingest(
|
||||
session_id="agent_sess_123",
|
||||
tool_pairs=[
|
||||
{"tool_name": "search", "tool_call_id": "call_1", "params": {"q": "..."}, "result": "...", "timestamp": "..."},
|
||||
],
|
||||
)
|
||||
|
||||
# Offload v2: server-side context compaction (sync wait for result)
|
||||
compacted = client.offload_compact(
|
||||
session_id="agent_sess_123",
|
||||
messages=[...],
|
||||
ratio=0.7,
|
||||
context_window=128000,
|
||||
)
|
||||
print(compacted["messages"], compacted["report"])
|
||||
|
||||
# Read memory pipeline artifacts (e.g. persona.md, scene_blocks/*.md)
|
||||
raw = client.read_file("scene_blocks/工作.md")
|
||||
```
|
||||
@@ -103,6 +120,9 @@ asyncio.run(main())
|
||||
| L2 | `rm_scenario()` | `POST /v2/scenario/rm` |
|
||||
| L3 | `read_core()` | `POST /v2/core/read` |
|
||||
| L3 | `write_core()` | `POST /v2/core/write` |
|
||||
| Offload | `offload_ingest()` | `POST /v2/offload/ingest` |
|
||||
| Offload | `offload_compact()` | `POST /v2/offload/compact` |
|
||||
| Offload | `offload_query_mmd()` | `POST /v2/offload/query-mmd` |
|
||||
|
||||
## Error Handling
|
||||
|
||||
|
||||
@@ -63,6 +63,23 @@ print(core["content"])
|
||||
# L3: 写入核心记忆
|
||||
client.write_core("# User Profile\n...")
|
||||
|
||||
# Offload v2: 上报工具调用对,触发服务端 L1 异步处理(可 fire-and-forget)
|
||||
client.offload_ingest(
|
||||
session_id="agent_sess_123",
|
||||
tool_pairs=[
|
||||
{"tool_name": "search", "tool_call_id": "call_1", "params": {"q": "..."}, "result": "...", "timestamp": "..."},
|
||||
],
|
||||
)
|
||||
|
||||
# Offload v2: 服务端上下文压缩(同步等待结果)
|
||||
compacted = client.offload_compact(
|
||||
session_id="agent_sess_123",
|
||||
messages=[...],
|
||||
ratio=0.7,
|
||||
context_window=128000,
|
||||
)
|
||||
print(compacted["messages"], compacted["report"])
|
||||
|
||||
# 读取记忆 pipeline 产物(如 persona.md、scene_blocks/*.md)
|
||||
raw = client.read_file("scene_blocks/工作.md")
|
||||
```
|
||||
@@ -103,6 +120,9 @@ asyncio.run(main())
|
||||
| L2 | `rm_scenario()` | `POST /v2/scenario/rm` |
|
||||
| L3 | `read_core()` | `POST /v2/core/read` |
|
||||
| L3 | `write_core()` | `POST /v2/core/write` |
|
||||
| Offload | `offload_ingest()` | `POST /v2/offload/ingest` |
|
||||
| Offload | `offload_compact()` | `POST /v2/offload/compact` |
|
||||
| Offload | `offload_query_mmd()` | `POST /v2/offload/query-mmd` |
|
||||
|
||||
## 错误处理
|
||||
|
||||
|
||||
@@ -255,6 +255,116 @@ class MemoryClient:
|
||||
"""``POST /core/write``"""
|
||||
return self._stub.post(f"{_V2}/core/write", {"content": content})
|
||||
|
||||
# -- Offload (Ingest + Compact + Query-MMD) ----------------------------
|
||||
|
||||
def offload_ingest(
|
||||
self,
|
||||
session_id: str,
|
||||
tool_pairs: List[Dict[str, Any]],
|
||||
*,
|
||||
prompt: Optional[str] = None,
|
||||
recent_messages: Optional[List[Dict[str, Any]]] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""``POST /v2/offload/ingest`` — 上报工具调用对,触发 L1 异步处理。
|
||||
|
||||
可 fire-and-forget 使用(忽略返回值)。
|
||||
|
||||
Parameters
|
||||
----------
|
||||
session_id : str
|
||||
会话 ID。
|
||||
tool_pairs : list[dict]
|
||||
工具调用对列表,每个元素包含 ``tool_name``、``tool_call_id``、
|
||||
``params``、``result``、``timestamp``,可选 ``duration_ms``。
|
||||
prompt : str, optional
|
||||
最新 user message,用于 L1.5 任务判断。
|
||||
recent_messages : list[dict], optional
|
||||
近期历史消息列表(``role`` + ``content``),辅助 L1 提取上下文。
|
||||
"""
|
||||
return self._stub.post(
|
||||
f"{_V2}/offload/ingest",
|
||||
_strip_none({
|
||||
"session_id": session_id,
|
||||
"tool_pairs": tool_pairs,
|
||||
"prompt": prompt,
|
||||
"recent_messages": recent_messages,
|
||||
}),
|
||||
)
|
||||
|
||||
def offload_compact(
|
||||
self,
|
||||
session_id: str,
|
||||
messages: List[Dict[str, Any]],
|
||||
ratio: float,
|
||||
total_tokens: int,
|
||||
*,
|
||||
context_window: Optional[int] = None,
|
||||
message_tokens: Optional[List[int]] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""``POST /v2/offload/compact`` — 对 messages 执行服务端上下文压缩。
|
||||
|
||||
Parameters
|
||||
----------
|
||||
session_id : str
|
||||
会话 ID。
|
||||
messages : list[dict]
|
||||
当前完整对话消息列表。
|
||||
ratio : float
|
||||
当前 token 使用比例(已用 / context_window),触发压缩策略判断。
|
||||
total_tokens : int
|
||||
当前完整上下文的总 token 数(包含 system prompt、tool schemas 等不在
|
||||
messages 中的隐性开销)。服务端用于计算 fixed overhead 和校准 token 估算。
|
||||
context_window : int, optional
|
||||
模型 context window 大小(token 数)。
|
||||
message_tokens : list[int], optional
|
||||
每条消息对应的 token 数,提供时可跳过服务端估算,提升性能。
|
||||
|
||||
Returns
|
||||
-------
|
||||
dict
|
||||
``messages``(压缩后消息列表)+ ``report``(压缩报告)。
|
||||
"""
|
||||
return self._stub.post(
|
||||
f"{_V2}/offload/compact",
|
||||
_strip_none({
|
||||
"session_id": session_id,
|
||||
"messages": messages,
|
||||
"ratio": ratio,
|
||||
"total_tokens": total_tokens,
|
||||
"context_window": context_window,
|
||||
"message_tokens": message_tokens,
|
||||
}),
|
||||
)
|
||||
|
||||
def offload_query_mmd(
|
||||
self,
|
||||
session_id: str,
|
||||
*,
|
||||
limit: Optional[int] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""``POST /v2/offload/query-mmd`` — 查询 session 的任务流程图(MMD 文件)。
|
||||
|
||||
Parameters
|
||||
----------
|
||||
session_id : str
|
||||
会话 ID。
|
||||
limit : int, optional
|
||||
最多返回几个 MMD 文件。``limit=1`` 时走快速路径只返回当前活跃 MMD。
|
||||
|
||||
Returns
|
||||
-------
|
||||
dict
|
||||
``mmds``(列表,每项含 ``filename``、``content``、``version``)+
|
||||
``current_mmd``(当前活跃 MMD 文件名,无则为 ``None``)。
|
||||
"""
|
||||
return self._stub.post(
|
||||
f"{_V2}/offload/query-mmd",
|
||||
_strip_none({
|
||||
"session_id": session_id,
|
||||
"limit": limit,
|
||||
}),
|
||||
)
|
||||
|
||||
# -- File read (memory pipeline artifacts) -----------------------------
|
||||
|
||||
def read_file(self, path: str) -> str:
|
||||
@@ -432,6 +542,65 @@ class AsyncMemoryClient:
|
||||
async def write_core(self, content: str) -> Dict[str, Any]:
|
||||
return await self._stub.post(f"{_V2}/core/write", {"content": content})
|
||||
|
||||
# -- Offload (Ingest + Compact + Query-MMD) ----------------------------
|
||||
|
||||
async def offload_ingest(
|
||||
self,
|
||||
session_id: str,
|
||||
tool_pairs: List[Dict[str, Any]],
|
||||
*,
|
||||
prompt: Optional[str] = None,
|
||||
recent_messages: Optional[List[Dict[str, Any]]] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""``POST /v2/offload/ingest``(异步)"""
|
||||
return await self._stub.post(
|
||||
f"{_V2}/offload/ingest",
|
||||
_strip_none({
|
||||
"session_id": session_id,
|
||||
"tool_pairs": tool_pairs,
|
||||
"prompt": prompt,
|
||||
"recent_messages": recent_messages,
|
||||
}),
|
||||
)
|
||||
|
||||
async def offload_compact(
|
||||
self,
|
||||
session_id: str,
|
||||
messages: List[Dict[str, Any]],
|
||||
ratio: float,
|
||||
total_tokens: int,
|
||||
*,
|
||||
context_window: Optional[int] = None,
|
||||
message_tokens: Optional[List[int]] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""``POST /v2/offload/compact``(异步)"""
|
||||
return await self._stub.post(
|
||||
f"{_V2}/offload/compact",
|
||||
_strip_none({
|
||||
"session_id": session_id,
|
||||
"messages": messages,
|
||||
"ratio": ratio,
|
||||
"total_tokens": total_tokens,
|
||||
"context_window": context_window,
|
||||
"message_tokens": message_tokens,
|
||||
}),
|
||||
)
|
||||
|
||||
async def offload_query_mmd(
|
||||
self,
|
||||
session_id: str,
|
||||
*,
|
||||
limit: Optional[int] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""``POST /v2/offload/query-mmd``(异步)"""
|
||||
return await self._stub.post(
|
||||
f"{_V2}/offload/query-mmd",
|
||||
_strip_none({
|
||||
"session_id": session_id,
|
||||
"limit": limit,
|
||||
}),
|
||||
)
|
||||
|
||||
# -- lifecycle ---------------------------------------------------------
|
||||
|
||||
# -- File read (memory pipeline artifacts) -----------------------------
|
||||
|
||||
@@ -58,6 +58,28 @@ console.log(core.content);
|
||||
// L3: write core memory
|
||||
await client.writeCore({ content: "# User Profile\n..." });
|
||||
|
||||
// Offload v2: send tool pairs for server-side L1 async processing (fire-and-forget)
|
||||
await client.offloadIngest({
|
||||
session_id: "agent_sess_123",
|
||||
tool_pairs: [
|
||||
{ tool_name: "search", tool_call_id: "call_1", params: { q: "..." }, result: "...", timestamp: "..." },
|
||||
],
|
||||
});
|
||||
|
||||
// Offload v2: server-side context compaction (sync wait for result)
|
||||
const compacted = await client.offloadCompact({
|
||||
session_id: "agent_sess_123",
|
||||
messages: [...],
|
||||
ratio: 0.7,
|
||||
context_window: 128000,
|
||||
total_tokens: 160000,
|
||||
});
|
||||
console.log(compacted.messages, compacted.report);
|
||||
|
||||
// Offload v2: query MMD task graphs
|
||||
const mmd = await client.offloadQueryMmd({ session_id: "agent_sess_123", limit: 1 });
|
||||
console.log(mmd.current_mmd, mmd.mmds);
|
||||
|
||||
// Read memory pipeline artifacts (e.g. persona.md, scene_blocks/*.md)
|
||||
const raw = await client.readFile("scene_blocks/工作.md");
|
||||
```
|
||||
@@ -80,6 +102,9 @@ const raw = await client.readFile("scene_blocks/工作.md");
|
||||
| L2 | `rmScenario()` | `POST /v2/scenario/rm` |
|
||||
| L3 | `readCore()` | `POST /v2/core/read` |
|
||||
| L3 | `writeCore()` | `POST /v2/core/write` |
|
||||
| Offload | `offloadIngest()` | `POST /v2/offload/ingest` |
|
||||
| Offload | `offloadCompact()` | `POST /v2/offload/compact` |
|
||||
| Offload | `offloadQueryMmd()` | `POST /v2/offload/query-mmd` |
|
||||
|
||||
## Error Handling
|
||||
|
||||
|
||||
@@ -58,6 +58,28 @@ console.log(core.content);
|
||||
// L3: 写入核心记忆
|
||||
await client.writeCore({ content: "# User Profile\n..." });
|
||||
|
||||
// Offload v2: 上报工具调用对,触发服务端 L1 异步处理(fire-and-forget)
|
||||
await client.offloadIngest({
|
||||
session_id: "agent_sess_123",
|
||||
tool_pairs: [
|
||||
{ tool_name: "search", tool_call_id: "call_1", params: { q: "..." }, result: "...", timestamp: "..." },
|
||||
],
|
||||
});
|
||||
|
||||
// Offload v2: 服务端上下文压缩(同步等待结果)
|
||||
const compacted = await client.offloadCompact({
|
||||
session_id: "agent_sess_123",
|
||||
messages: [...],
|
||||
ratio: 0.7,
|
||||
context_window: 128000,
|
||||
total_tokens: 160000,
|
||||
});
|
||||
console.log(compacted.messages, compacted.report);
|
||||
|
||||
// Offload v2: 查询任务流程图(MMD)
|
||||
const mmd = await client.offloadQueryMmd({ session_id: "agent_sess_123", limit: 1 });
|
||||
console.log(mmd.current_mmd, mmd.mmds);
|
||||
|
||||
// 读取记忆 pipeline 产物(如 persona.md、scene_blocks/*.md)
|
||||
const raw = await client.readFile("scene_blocks/工作.md");
|
||||
```
|
||||
@@ -80,6 +102,9 @@ const raw = await client.readFile("scene_blocks/工作.md");
|
||||
| L2 | `rmScenario()` | `POST /v2/scenario/rm` |
|
||||
| L3 | `readCore()` | `POST /v2/core/read` |
|
||||
| L3 | `writeCore()` | `POST /v2/core/write` |
|
||||
| Offload | `offloadIngest()` | `POST /v2/offload/ingest` |
|
||||
| Offload | `offloadCompact()` | `POST /v2/offload/compact` |
|
||||
| Offload | `offloadQueryMmd()` | `POST /v2/offload/query-mmd` |
|
||||
|
||||
## 错误处理
|
||||
|
||||
|
||||
@@ -25,6 +25,12 @@ import type {
|
||||
CoreFile,
|
||||
CoreWriteData,
|
||||
CoreWriteRequest,
|
||||
OffloadCompactData,
|
||||
OffloadCompactRequest,
|
||||
OffloadIngestData,
|
||||
OffloadIngestRequest,
|
||||
OffloadQueryMmdData,
|
||||
OffloadQueryMmdRequest,
|
||||
ScenarioFile,
|
||||
ScenarioListData,
|
||||
ScenarioListRequest,
|
||||
@@ -148,6 +154,32 @@ export class MemoryClient {
|
||||
return this.http.post(`${V2}/core/write`, params as unknown as Record<string, unknown>);
|
||||
}
|
||||
|
||||
// -- Offload (Compaction + Ingest) ------------------------------------
|
||||
|
||||
/**
|
||||
* Send tool pairs (+ optional context) to offload server for L1 processing.
|
||||
* Fire-and-forget usage: caller can `.catch()` without blocking.
|
||||
*/
|
||||
offloadIngest(params: OffloadIngestRequest): Promise<OffloadIngestData> {
|
||||
return this.http.post(`${V2}/offload/ingest`, stripUndefined(params as unknown as Record<string, unknown>));
|
||||
}
|
||||
|
||||
/**
|
||||
* Request server-side context compaction.
|
||||
* Returns compacted messages + report, or throws on failure.
|
||||
*/
|
||||
offloadCompact(params: OffloadCompactRequest): Promise<OffloadCompactData> {
|
||||
return this.http.post(`${V2}/offload/compact`, stripUndefined(params as unknown as Record<string, unknown>));
|
||||
}
|
||||
|
||||
/**
|
||||
* Query MMD task graphs for a session.
|
||||
* limit=1 returns only the current active MMD (fast path).
|
||||
*/
|
||||
offloadQueryMmd(params: OffloadQueryMmdRequest): Promise<OffloadQueryMmdData> {
|
||||
return this.http.post(`${V2}/offload/query-mmd`, stripUndefined(params as unknown as Record<string, unknown>));
|
||||
}
|
||||
|
||||
// -- File read (memory pipeline artifacts) ----------------------------
|
||||
|
||||
/**
|
||||
|
||||
@@ -6,4 +6,28 @@ export { MemoryClient, type MemoryClientConfig, type Transport } from "./client.
|
||||
export { TDAMError } from "./errors.js";
|
||||
export { HttpTransport, type HttpTransportOptions } from "./http.js";
|
||||
export { MemoryFileReader, StsCredentialManager, StsCredential, createMemoryFileReader, cosV5Sign, type MemoryFileReaderConfig } from "./cos.js";
|
||||
export type * from "./types.js";
|
||||
export type {
|
||||
// L0
|
||||
ConversationItem, ConversationAddRequest, ConversationAddData,
|
||||
ConversationQueryRequest, ConversationQueryData,
|
||||
ConversationSearchRequest, ConversationSearchData,
|
||||
ConversationDeleteRequest, ConversationDeleteData,
|
||||
// L1
|
||||
AtomicDetail, AtomicUpdateRequest, AtomicUpdateData,
|
||||
AtomicQueryRequest, AtomicQueryData,
|
||||
AtomicSearchRequest, AtomicSearchData,
|
||||
AtomicDeleteRequest, AtomicDeleteData,
|
||||
// L2
|
||||
ScenarioEntry, ScenarioListRequest, ScenarioListData,
|
||||
ScenarioReadRequest, ScenarioFile,
|
||||
ScenarioWriteRequest, ScenarioWriteData,
|
||||
ScenarioRmRequest,
|
||||
// L3
|
||||
CoreFile, CoreWriteRequest, CoreWriteData,
|
||||
// Offload
|
||||
OffloadToolPair, OffloadRecentMessage,
|
||||
OffloadIngestRequest, OffloadIngestData,
|
||||
OffloadCompactRequest, OffloadCompactData, OffloadCompactReport,
|
||||
// Common
|
||||
ApiResponseEnvelope,
|
||||
} from "./types.js";
|
||||
|
||||
@@ -188,3 +188,75 @@ export interface CoreWriteRequest {
|
||||
export interface CoreWriteData {
|
||||
updated_at: string;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Offload (Compaction + Ingest)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface OffloadToolPair {
|
||||
tool_name: string;
|
||||
tool_call_id: string;
|
||||
params: unknown;
|
||||
result: unknown;
|
||||
error?: string;
|
||||
timestamp: string;
|
||||
duration_ms?: number;
|
||||
}
|
||||
|
||||
export interface OffloadRecentMessage {
|
||||
role: "user" | "assistant";
|
||||
content: string;
|
||||
}
|
||||
|
||||
export interface OffloadIngestRequest {
|
||||
session_id: string;
|
||||
tool_pairs: OffloadToolPair[];
|
||||
prompt?: string;
|
||||
recent_messages?: OffloadRecentMessage[];
|
||||
}
|
||||
|
||||
export interface OffloadIngestData {
|
||||
accepted: boolean;
|
||||
}
|
||||
|
||||
export interface OffloadCompactRequest {
|
||||
session_id: string;
|
||||
messages: unknown[];
|
||||
ratio: number;
|
||||
context_window: number;
|
||||
total_tokens: number;
|
||||
message_tokens?: number[];
|
||||
}
|
||||
|
||||
export interface OffloadCompactReport {
|
||||
resolvedLevel: string;
|
||||
originalCount: number;
|
||||
compactedCount: number;
|
||||
fastPathReplaced: number;
|
||||
fastPathDeleted: number;
|
||||
mildReplacements: number;
|
||||
aggressiveDeleted: number;
|
||||
emergencyDeleted: number;
|
||||
mmdInjected: number;
|
||||
}
|
||||
|
||||
export interface OffloadCompactData {
|
||||
messages: unknown[];
|
||||
report: OffloadCompactReport;
|
||||
}
|
||||
|
||||
export interface OffloadQueryMmdRequest {
|
||||
session_id: string;
|
||||
limit?: number;
|
||||
}
|
||||
|
||||
export interface OffloadMmdFile {
|
||||
filename: string;
|
||||
content: string;
|
||||
version: number;
|
||||
}
|
||||
|
||||
export interface OffloadQueryMmdData {
|
||||
mmds: OffloadMmdFile[];
|
||||
current_mmd: string | null;
|
||||
}
|
||||
|
||||
+38
-4
@@ -213,11 +213,12 @@ export interface OffloadConfig {
|
||||
* LLM execution mode for L1/L1.5/L2 tasks.
|
||||
* - "local": call LLM directly via AI SDK (uses offload.model or main agent model)
|
||||
* - "backend": route through remote backend service (requires backendUrl)
|
||||
* - "client": stateless client mode — all compression delegated to offload server v2
|
||||
* - "collect": data collection only — runs L1/L1.5/L2 asynchronously but disables
|
||||
* L3 compression and does NOT occupy the contextEngine slot (uses legacy compaction)
|
||||
* Default: "local" (auto-detects based on backendUrl presence for backward compat)
|
||||
*/
|
||||
mode: "local" | "backend" | "collect";
|
||||
mode: "local" | "backend" | "client" | "collect";
|
||||
/** LLM model for offload tasks, format: "provider/model-id". Falls back to agents.defaults.model when omitted. */
|
||||
model?: string;
|
||||
/** LLM temperature (default: 0.2) */
|
||||
@@ -265,6 +266,22 @@ export interface OffloadConfig {
|
||||
* primary non-loopback IPv4 address.
|
||||
*/
|
||||
userId?: string;
|
||||
|
||||
// ── Client mode fields (used when mode === "client") ──────────────
|
||||
/** Offload server v2 base URL (e.g. "http://localhost:9100"). */
|
||||
serverUrl?: string;
|
||||
/** Bearer token for offload server v2 Authorization header. */
|
||||
apiKey?: string;
|
||||
/** X-TDAI-Service-Id header value. */
|
||||
serviceId?: string;
|
||||
/** Agent name used in storage path (default: "default"). */
|
||||
agentName?: string;
|
||||
/** Client-side threshold: skip compaction when ratio < this value (default: 0.5). */
|
||||
compactionRatio?: number;
|
||||
/** Ingest request timeout in ms (default: 5000). */
|
||||
ingestTimeoutMs?: number;
|
||||
/** Compaction request timeout in ms (default: 30000). */
|
||||
compactionTimeoutMs?: number;
|
||||
}
|
||||
|
||||
/** Fully resolved plugin configuration (v3). */
|
||||
@@ -439,10 +456,20 @@ export function parseConfig(raw: Record<string, unknown> | undefined): MemoryTda
|
||||
// --- Offload ---
|
||||
const offloadGroup = obj(c, "offload");
|
||||
|
||||
const offloadMode: "local" | "backend" | "collect" = (() => {
|
||||
// Auto-derive offload serverUrl/apiKey/serviceId from top-level server config
|
||||
// when offload client fields are not explicitly set.
|
||||
const serverGroup = obj(c, "server");
|
||||
const serverUrl = optStr(serverGroup, "url");
|
||||
const serverApiKey = optStr(serverGroup, "apiKey");
|
||||
const serverInstanceId = optStr(serverGroup, "instanceId");
|
||||
|
||||
const offloadMode: "local" | "backend" | "client" | "collect" = (() => {
|
||||
const raw = optStr(offloadGroup, "mode");
|
||||
if (raw === "local" || raw === "backend" || raw === "collect") return raw;
|
||||
return optStr(offloadGroup, "backendUrl") ? "backend" : "local";
|
||||
if (raw === "local" || raw === "backend" || raw === "client" || raw === "collect") return raw;
|
||||
// Auto-derive: if backendUrl is set → "backend"; if server.url is set → "client"; else "local"
|
||||
if (optStr(offloadGroup, "backendUrl")) return "backend";
|
||||
if (serverUrl) return "client";
|
||||
return "local";
|
||||
})();
|
||||
|
||||
const offload: OffloadConfig = {
|
||||
@@ -465,6 +492,13 @@ export function parseConfig(raw: Record<string, unknown> | undefined): MemoryTda
|
||||
offloadRetentionDays: normalizeOffloadRetentionDays(num(offloadGroup, "offloadRetentionDays") ?? 0),
|
||||
logMaxSizeMb: num(offloadGroup, "logMaxSizeMb") ?? 50,
|
||||
userId: optStr(offloadGroup, "userId"),
|
||||
// Client mode fields — fall back to top-level server config when not explicitly set
|
||||
serverUrl: optStr(offloadGroup, "serverUrl") ?? serverUrl,
|
||||
apiKey: optStr(offloadGroup, "apiKey") ?? serverApiKey,
|
||||
serviceId: optStr(offloadGroup, "serviceId") ?? serverInstanceId,
|
||||
compactionRatio: num(offloadGroup, "compactionRatio") ?? 0.5,
|
||||
ingestTimeoutMs: num(offloadGroup, "ingestTimeoutMs") ?? 5000,
|
||||
compactionTimeoutMs: num(offloadGroup, "compactionTimeoutMs") ?? 30000,
|
||||
};
|
||||
|
||||
return {
|
||||
|
||||
@@ -18,6 +18,7 @@ import crypto from "node:crypto";
|
||||
import { sanitizeText, stripCodeBlocks, shouldCaptureL0 } from "../../utils/sanitize.js";
|
||||
import type { StorageAdapter } from "../storage/adapter.js";
|
||||
import { StoragePaths } from "../storage/types.js";
|
||||
import type { Logger } from "../types.js";
|
||||
|
||||
// ============================
|
||||
// Types
|
||||
@@ -63,13 +64,6 @@ export interface L0ConversationRecord {
|
||||
messages: ConversationMessage[];
|
||||
}
|
||||
|
||||
interface Logger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
|
||||
const TAG = "[memory-tdai][l0]";
|
||||
|
||||
// ============================
|
||||
|
||||
@@ -20,14 +20,9 @@ import type { IMemoryStore, L0Record } from "../store/types.js";
|
||||
import type { EmbeddingService } from "../store/embedding.js";
|
||||
import type { StorageAdapter } from "../storage/adapter.js";
|
||||
|
||||
const TAG = "[memory-tdai] [capture]";
|
||||
import type { Logger } from "../types.js";
|
||||
|
||||
interface Logger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
const TAG = "[memory-tdai] [capture]";
|
||||
|
||||
export interface AutoCaptureResult {
|
||||
/** Whether the scheduler was notified (conversation count incremented) */
|
||||
|
||||
@@ -21,6 +21,7 @@ import type { EmbeddingService, EmbeddingCallOptions } from "../store/embedding.
|
||||
import { sanitizeText } from "../../utils/sanitize.js";
|
||||
import type { StorageAdapter } from "../storage/adapter.js";
|
||||
import { StoragePaths } from "../storage/types.js";
|
||||
import type { Logger } from "../types.js";
|
||||
|
||||
const TAG = "[memory-tdai] [recall]";
|
||||
const RECALL_TRUNCATION_SUFFIX = "…(已截断;可用 tdai_memory_search 或 tdai_conversation_search 查看详情)";
|
||||
@@ -46,13 +47,6 @@ const MEMORY_TOOLS_GUIDE = `<memory-tools-guide>
|
||||
- 若 3 次搜索后仍无结果,说明该信息不在记忆中,请直接根据已有信息回复用户,不要继续搜索。
|
||||
</memory-tools-guide>`
|
||||
|
||||
interface Logger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
|
||||
/** A single recalled L1 memory with its search score and type. */
|
||||
export interface RecalledMemory {
|
||||
content: string;
|
||||
@@ -890,11 +884,15 @@ function normalizeBudgetLimit(value: number | undefined): number | undefined {
|
||||
}
|
||||
|
||||
function truncateRecallLine(line: string, maxChars: number): string {
|
||||
if (line.length <= maxChars) return line;
|
||||
// Count and slice by code point, not UTF-16 code unit, so a cut never lands
|
||||
// between the halves of a surrogate pair (which would corrupt a non-BMP
|
||||
// character to U+FFFD when the line is UTF-8 encoded for the request).
|
||||
const cps = Array.from(line);
|
||||
if (cps.length <= maxChars) return line;
|
||||
if (maxChars <= RECALL_TRUNCATION_SUFFIX.length) {
|
||||
return line.slice(0, maxChars);
|
||||
return cps.slice(0, maxChars).join("");
|
||||
}
|
||||
return `${line.slice(0, maxChars - RECALL_TRUNCATION_SUFFIX.length).trimEnd()}${RECALL_TRUNCATION_SUFFIX}`;
|
||||
return `${cps.slice(0, maxChars - RECALL_TRUNCATION_SUFFIX.length).join("").trimEnd()}${RECALL_TRUNCATION_SUFFIX}`;
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -12,19 +12,12 @@ import { BackupManager } from "../../utils/backup.js";
|
||||
import { escapeXmlTags } from "../../utils/sanitize.js";
|
||||
import { report } from "../report/reporter.js";
|
||||
import { reportL3LatencyMetrics } from "../report/metric-tracking-l3-latency.js";
|
||||
import type { LLMRunner } from "../types.js";
|
||||
import type { LLMRunner, Logger } from "../types.js";
|
||||
import type { StorageAdapter } from "../storage/adapter.js";
|
||||
import { StoragePaths } from "../storage/types.js";
|
||||
|
||||
const TAG = "[memory-tdai] [persona]";
|
||||
|
||||
interface Logger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
|
||||
export class PersonaGenerator {
|
||||
private dataDir: string;
|
||||
private runner: LLMRunner;
|
||||
|
||||
@@ -8,14 +8,11 @@ import { stripSceneNavigation } from "../scene/scene-navigation.js";
|
||||
import type { StorageAdapter } from "../storage/adapter.js";
|
||||
import { StoragePaths } from "../storage/types.js";
|
||||
|
||||
import type { Logger } from "../types.js";
|
||||
|
||||
const TAG = "[memory-tdai] [trigger]";
|
||||
|
||||
interface TriggerLogger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
type TriggerLogger = Logger;
|
||||
|
||||
export interface TriggerResult {
|
||||
should: boolean;
|
||||
|
||||
@@ -6,6 +6,7 @@ import { readSceneIndex, syncSceneIndex } from "../scene/scene-index.js";
|
||||
import { generateSceneNavigation, stripSceneNavigation } from "../scene/scene-navigation.js";
|
||||
import type { StorageAdapter } from "../storage/adapter.js";
|
||||
import { StoragePaths } from "../storage/types.js";
|
||||
import type { Logger } from "../types.js";
|
||||
|
||||
const PROFILE_SCOPE = "global";
|
||||
|
||||
@@ -15,13 +16,6 @@ function isRenameRaceError(err: unknown): boolean {
|
||||
return code === "ENOTEMPTY" || code === "EEXIST";
|
||||
}
|
||||
|
||||
interface Logger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
|
||||
export interface ProfileBaseline {
|
||||
version: number;
|
||||
contentMd5: string;
|
||||
|
||||
@@ -19,14 +19,7 @@ import { sanitizeJsonForParse } from "../../utils/sanitize.js";
|
||||
import type { IMemoryStore } from "../store/types.js";
|
||||
import { buildFtsQuery } from "../store/sqlite.js";
|
||||
import type { EmbeddingService } from "../store/embedding.js";
|
||||
import type { LLMRunner } from "../types.js";
|
||||
|
||||
interface Logger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
import type { LLMRunner, Logger } from "../types.js";
|
||||
|
||||
const TAG = "[memory-tdai][l1-dedup]";
|
||||
|
||||
|
||||
@@ -24,16 +24,9 @@ import type { EmbeddingService } from "../store/embedding.js";
|
||||
import { report } from "../report/reporter.js";
|
||||
import { metricProducer } from "../report/kafka-metric-producer.js";
|
||||
import { reportL1LatencyMetrics } from "../report/metric-tracking-l1-latency.js";
|
||||
import type { LLMRunner } from "../types.js";
|
||||
import type { LLMRunner, Logger } from "../types.js";
|
||||
import type { StorageAdapter } from "../storage/adapter.js";
|
||||
|
||||
interface Logger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
|
||||
const TAG = "[memory-tdai][l1-extractor]";
|
||||
|
||||
// ============================
|
||||
|
||||
@@ -19,13 +19,7 @@ import { StoragePaths } from "../storage/types.js";
|
||||
// Re-export types that readers need
|
||||
export type { MemoryRecord, MemoryType, EpisodicMetadata } from "./l1-writer.js";
|
||||
export type { L1QueryFilter } from "../store/types.js";
|
||||
|
||||
interface Logger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
import type { Logger } from "../types.js";
|
||||
|
||||
const TAG = "[memory-tdai] [l1-reader]";
|
||||
|
||||
|
||||
@@ -21,6 +21,7 @@ import type { IMemoryStore } from "../store/types.js";
|
||||
import type { EmbeddingService } from "../store/embedding.js";
|
||||
import type { StorageAdapter } from "../storage/adapter.js";
|
||||
import { StoragePaths } from "../storage/types.js";
|
||||
import type { Logger } from "../types.js";
|
||||
|
||||
// ============================
|
||||
// Types
|
||||
@@ -110,13 +111,6 @@ export interface DedupDecision {
|
||||
merged_timestamps?: string[];
|
||||
}
|
||||
|
||||
interface Logger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
|
||||
const TAG = "[memory-tdai][l1-writer]";
|
||||
|
||||
// ============================
|
||||
|
||||
@@ -198,14 +198,19 @@ export class MetricTrackingRunner implements LLMRunner {
|
||||
}
|
||||
|
||||
async run(params: LLMRunParams): Promise<string> {
|
||||
// 0. 注入 instanceId(如果调用方没传,从 getInstanceId 回调获取)
|
||||
const enrichedParams = params.instanceId
|
||||
? params
|
||||
: { ...params, instanceId: this.getInstanceId() };
|
||||
|
||||
// 1. 先执行原方法,拿到结果(异常直接 re-throw)
|
||||
const text = await this.inner.run(params);
|
||||
const text = await this.inner.run(enrichedParams);
|
||||
|
||||
// 2. 原方法成功后,try-catch 做上报(静默失败)
|
||||
try {
|
||||
const metricName = taskIdToMetricName(params.taskId);
|
||||
if (metricName) {
|
||||
const instanceId = params.instanceId ?? this.getInstanceId();
|
||||
const instanceId = enrichedParams.instanceId ?? this.getInstanceId();
|
||||
if (instanceId) {
|
||||
// 优先从 inner runner 的 lastUsage side-channel 读取精确 token 数
|
||||
const innerWithUsage = this.inner as LLMRunnerWithUsage;
|
||||
|
||||
@@ -62,6 +62,24 @@ export class TracedTaskExecutor implements TaskExecutor {
|
||||
return this.executeL1(task);
|
||||
}
|
||||
|
||||
async executeOffloadL1?(task: TaskPayload, signal?: AbortSignal): Promise<void> {
|
||||
if (this.inner.executeOffloadL1) {
|
||||
return this.executeWithTrace("offload-l1", task, () => this.inner.executeOffloadL1!(task, signal));
|
||||
}
|
||||
}
|
||||
|
||||
async executeOffloadL15?(task: TaskPayload, signal?: AbortSignal): Promise<void> {
|
||||
if (this.inner.executeOffloadL15) {
|
||||
return this.executeWithTrace("offload-l15", task, () => this.inner.executeOffloadL15!(task, signal));
|
||||
}
|
||||
}
|
||||
|
||||
async executeOffloadL2?(task: TaskPayload, signal?: AbortSignal): Promise<void> {
|
||||
if (this.inner.executeOffloadL2) {
|
||||
return this.executeWithTrace("offload-l2", task, () => this.inner.executeOffloadL2!(task, signal));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 核心方法:在 Trace Context 中执行任务。
|
||||
*
|
||||
|
||||
@@ -27,18 +27,13 @@ import { normalizeSceneFilenames } from "./filename-normalizer.js";
|
||||
import { buildSceneExtractionPrompt } from "../prompts/scene-extraction.js";
|
||||
import { report } from "../report/reporter.js";
|
||||
import { reportL2LatencyMetrics } from "../report/metric-tracking-l2-latency.js";
|
||||
import type { LLMRunner } from "../types.js";
|
||||
import type { LLMRunner, Logger } from "../types.js";
|
||||
import type { StorageAdapter } from "../storage/adapter.js";
|
||||
import { StoragePaths } from "../storage/types.js";
|
||||
|
||||
const TAG = "[memory-tdai] [extractor]";
|
||||
|
||||
interface ExtractorLogger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
type ExtractorLogger = Logger;
|
||||
|
||||
export interface ExtractionResult {
|
||||
memoriesProcessed: number;
|
||||
@@ -470,24 +465,24 @@ export class SceneExtractor {
|
||||
success: true,
|
||||
error: null,
|
||||
});
|
||||
}
|
||||
|
||||
// ── 评测指标:L2 延迟 + 场景变化 ──
|
||||
try {
|
||||
const postSceneCount = (await readSceneIndex(this.dataDir, this.storage)).length;
|
||||
reportL2LatencyMetrics({
|
||||
instanceId: this.instanceId ?? "",
|
||||
extractionLatencyMs: totalMs,
|
||||
llmDurationMs: llmDurationMs > 0 ? llmDurationMs : null,
|
||||
sceneCountBefore: preExtractIndex.size,
|
||||
sceneCountAfter: postSceneCount,
|
||||
scenesCreated,
|
||||
scenesUpdated,
|
||||
scenesDeleted,
|
||||
hasError: false,
|
||||
});
|
||||
} catch {
|
||||
// 静默忽略
|
||||
// ── 评测指标:L2 延迟 + 场景变化 ──
|
||||
try {
|
||||
const postSceneCount = (await readSceneIndex(this.dataDir, this.storage)).length;
|
||||
reportL2LatencyMetrics({
|
||||
instanceId: this.instanceId ?? "",
|
||||
extractionLatencyMs: totalMs,
|
||||
llmDurationMs: llmDurationMs > 0 ? llmDurationMs : null,
|
||||
sceneCountBefore: preExtractIndex.size,
|
||||
sceneCountAfter: postSceneCount,
|
||||
scenesCreated,
|
||||
scenesUpdated,
|
||||
scenesDeleted,
|
||||
hasError: false,
|
||||
});
|
||||
} catch {
|
||||
// 静默忽略
|
||||
}
|
||||
}
|
||||
|
||||
// Detect empty extraction: pre and post scene_index both empty means LLM didn't write anything
|
||||
|
||||
@@ -56,7 +56,7 @@ export interface TimerEntry {
|
||||
|
||||
export interface TaskPayload {
|
||||
id: string;
|
||||
type: "L1" | "L2" | "L3" | "flush";
|
||||
type: "L1" | "L2" | "L3" | "flush" | "offload-l1" | "offload-l15" | "offload-l2";
|
||||
instanceId: string;
|
||||
sessionId: string;
|
||||
priority: number; // 0=high, 1=normal, 2=low
|
||||
|
||||
@@ -13,6 +13,8 @@
|
||||
* check health to dynamically downgrade to pure semantic search.
|
||||
*/
|
||||
|
||||
import type { Logger } from "../types.js";
|
||||
|
||||
// ============================
|
||||
// Types
|
||||
// ============================
|
||||
@@ -20,13 +22,6 @@
|
||||
/** Sparse vector: array of [token_hash, weight] pairs. */
|
||||
export type SparseVector = Array<[number, number]>;
|
||||
|
||||
interface Logger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
|
||||
export interface BM25ClientConfig {
|
||||
/** Sidecar service URL (default: "http://127.0.0.1:8084") */
|
||||
serviceUrl: string;
|
||||
|
||||
@@ -11,16 +11,10 @@
|
||||
|
||||
import { BM25Encoder } from "@tencentdb-agent-memory/tcvdb-text";
|
||||
import type { SparseVector } from "@tencentdb-agent-memory/tcvdb-text";
|
||||
import type { Logger } from "../types.js";
|
||||
|
||||
export type { SparseVector };
|
||||
|
||||
interface Logger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
|
||||
export interface BM25LocalConfig {
|
||||
/** Whether BM25 sparse encoding is enabled (default: true) */
|
||||
enabled: boolean;
|
||||
|
||||
@@ -13,6 +13,8 @@
|
||||
* - Throws on failure; callers decide fallback strategy.
|
||||
*/
|
||||
|
||||
import type { Logger } from "../types.js";
|
||||
|
||||
// ============================
|
||||
// Types
|
||||
// ============================
|
||||
@@ -104,17 +106,6 @@ export class EmbeddingNotReadyError extends Error {
|
||||
}
|
||||
}
|
||||
|
||||
// ============================
|
||||
// Logger interface
|
||||
// ============================
|
||||
|
||||
interface Logger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
|
||||
const TAG = "[memory-tdai][embedding]";
|
||||
|
||||
// ============================
|
||||
|
||||
@@ -21,6 +21,8 @@
|
||||
*/
|
||||
|
||||
import { createRequire } from "node:module";
|
||||
import { mkdirSync, existsSync } from "node:fs";
|
||||
import path from "node:path";
|
||||
import type { DatabaseSync, StatementSync, SQLInputValue } from "node:sqlite";
|
||||
import type { MemoryRecord } from "../record/l1-writer.js";
|
||||
import type { EmbeddingProviderInfo } from "./embedding.js";
|
||||
@@ -40,6 +42,9 @@ import type {
|
||||
L1PaginatedFilter,
|
||||
L1PaginatedResult,
|
||||
} from "./types.js";
|
||||
import type { Logger } from "../types.js";
|
||||
|
||||
export type { L1RecordRow } from "./types.js";
|
||||
|
||||
// ============================
|
||||
// Types
|
||||
@@ -86,13 +91,6 @@ export interface L0RecordRow {
|
||||
timestamp: number;
|
||||
}
|
||||
|
||||
interface Logger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
|
||||
const TAG = "[memory-tdai][sqlite]";
|
||||
|
||||
/** Persisted metadata about the embedding provider used to generate stored vectors. */
|
||||
@@ -394,6 +392,12 @@ export class VectorStore implements IMemoryStore {
|
||||
this.dimensions = dimensions;
|
||||
this.logger = logger;
|
||||
|
||||
// Ensure parent directory exists (for non-default instance paths)
|
||||
const dbDir = path.dirname(dbPath);
|
||||
if (!existsSync(dbDir)) {
|
||||
mkdirSync(dbDir, { recursive: true });
|
||||
}
|
||||
|
||||
// Open database with extension support enabled
|
||||
const { DatabaseSync: DbSync } = requireNodeSqlite();
|
||||
this.db = new DbSync(dbPath, { allowExtension: true });
|
||||
|
||||
@@ -16,6 +16,7 @@
|
||||
*/
|
||||
|
||||
import path from "node:path";
|
||||
import { existsSync, mkdirSync } from "node:fs";
|
||||
import type { MemoryTdaiConfig } from "../../config.js";
|
||||
import type { IMemoryStore, StoreLogger } from "./types.js";
|
||||
import type { EmbeddingService } from "./embedding.js";
|
||||
@@ -303,6 +304,11 @@ export class StorePool {
|
||||
|
||||
const dims = embCfg.dimensions ?? 0;
|
||||
const dbPath = this.getSqlitePath(instanceId);
|
||||
// 确保数据库目录存在(对于非 default instance)
|
||||
const dbDir = path.dirname(dbPath);
|
||||
if (!existsSync(dbDir)) {
|
||||
mkdirSync(dbDir, { recursive: true });
|
||||
}
|
||||
const store = new VectorStore(dbPath, dims, this.logger as StoreLogger);
|
||||
|
||||
return {
|
||||
|
||||
@@ -17,6 +17,7 @@
|
||||
|
||||
import type { MemoryRecord } from "../record/l1-writer.js";
|
||||
import type { EmbeddingProviderInfo } from "./embedding.js";
|
||||
import type { Logger } from "../types.js";
|
||||
|
||||
// Re-export so consumers can import everything from types.ts
|
||||
export type { MemoryRecord, EmbeddingProviderInfo };
|
||||
@@ -26,12 +27,7 @@ export type { MemoryRecord, EmbeddingProviderInfo };
|
||||
// ============================
|
||||
|
||||
/** Minimal logger interface accepted by store implementations. */
|
||||
export interface StoreLogger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
export type StoreLogger = Logger;
|
||||
|
||||
// ============================
|
||||
// L1 Types (Structured Memories)
|
||||
|
||||
@@ -13,18 +13,12 @@
|
||||
import type { IMemoryStore, L0SearchResult } from "../store/types.js";
|
||||
import { buildFtsQuery } from "../store/sqlite.js";
|
||||
import type { EmbeddingService } from "../store/embedding.js";
|
||||
import type { Logger } from "../types.js";
|
||||
|
||||
// ============================
|
||||
// Types
|
||||
// ============================
|
||||
|
||||
interface Logger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
|
||||
export interface ConversationSearchResultItem {
|
||||
id: string;
|
||||
session_key: string;
|
||||
|
||||
@@ -13,18 +13,12 @@
|
||||
import type { IMemoryStore, L1SearchResult } from "../store/types.js";
|
||||
import { buildFtsQuery } from "../store/sqlite.js";
|
||||
import type { EmbeddingService } from "../store/embedding.js";
|
||||
import type { Logger } from "../types.js";
|
||||
|
||||
// ============================
|
||||
// Types
|
||||
// ============================
|
||||
|
||||
interface Logger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
|
||||
export interface MemorySearchResultItem {
|
||||
id: string;
|
||||
content: string;
|
||||
|
||||
+3
-3
@@ -15,10 +15,10 @@
|
||||
// ============================
|
||||
|
||||
/**
|
||||
* Minimal logger interface used throughout TDAI Core.
|
||||
* Canonical logger interface used across all TDAI modules.
|
||||
*
|
||||
* Matches the existing `StoreLogger` and `RunnerLogger` interfaces
|
||||
* already used in the codebase — no migration needed for existing callers.
|
||||
* Named variants (StoreLogger, PluginLogger, etc.) are type aliases
|
||||
* of this interface, kept for backward compatibility.
|
||||
*/
|
||||
export interface Logger {
|
||||
debug?: (message: string) => void;
|
||||
|
||||
@@ -292,6 +292,25 @@ export interface GatewayConfig {
|
||||
cos: CosExtraConfig;
|
||||
/** 可观测性配置 (yaml: observability, env: KAFKA_METRIC_*) */
|
||||
observability: ObservabilityConfig;
|
||||
/** Offload server executor 配置 (yaml: offload) */
|
||||
offload: {
|
||||
forceTriggerThreshold: number;
|
||||
pendingMaxAgeSeconds: number;
|
||||
l1Temperature: number;
|
||||
l1MaxTokens: number;
|
||||
l1TimeoutMs: number;
|
||||
l15Temperature: number;
|
||||
l15MaxTokens: number;
|
||||
l15TimeoutMs: number;
|
||||
l2Temperature: number;
|
||||
l2MaxTokens: number;
|
||||
l2TimeoutMs: number;
|
||||
l2NullThreshold: number;
|
||||
mildOffloadRatio: number;
|
||||
aggressiveCompressRatio: number;
|
||||
emergencyCompressRatio: number;
|
||||
maxRetries: number;
|
||||
};
|
||||
}
|
||||
|
||||
// ============================
|
||||
@@ -491,6 +510,27 @@ export function loadGatewayConfig(overrides?: Partial<GatewayConfig>): GatewayCo
|
||||
|
||||
const observability: ObservabilityConfig = { otel, clickhouse, kafka, langfuse };
|
||||
|
||||
// Offload executor config (yaml: offload)
|
||||
const offloadConfig = obj(fileConfig, "offload");
|
||||
const offload = {
|
||||
forceTriggerThreshold: num(offloadConfig, "forceTriggerThreshold") ?? 4,
|
||||
pendingMaxAgeSeconds: num(offloadConfig, "pendingMaxAgeSeconds") ?? 30,
|
||||
l1Temperature: num(offloadConfig, "l1Temperature") ?? 0.3,
|
||||
l1MaxTokens: num(offloadConfig, "l1MaxTokens") ?? 8000,
|
||||
l1TimeoutMs: num(offloadConfig, "l1TimeoutMs") ?? 120_000,
|
||||
l15Temperature: num(offloadConfig, "l15Temperature") ?? 0.2,
|
||||
l15MaxTokens: num(offloadConfig, "l15MaxTokens") ?? 3000,
|
||||
l15TimeoutMs: num(offloadConfig, "l15TimeoutMs") ?? 120_000,
|
||||
l2Temperature: num(offloadConfig, "l2Temperature") ?? 0.4,
|
||||
l2MaxTokens: num(offloadConfig, "l2MaxTokens") ?? 16000,
|
||||
l2TimeoutMs: num(offloadConfig, "l2TimeoutMs") ?? 120_000,
|
||||
l2NullThreshold: num(offloadConfig, "l2NullThreshold") ?? 6,
|
||||
mildOffloadRatio: num(offloadConfig, "mildOffloadRatio") ?? 0.5,
|
||||
aggressiveCompressRatio: num(offloadConfig, "aggressiveCompressRatio") ?? 0.85,
|
||||
emergencyCompressRatio: num(offloadConfig, "emergencyCompressRatio") ?? 0.95,
|
||||
maxRetries: num(offloadConfig, "maxRetries") ?? 3,
|
||||
};
|
||||
|
||||
const base: GatewayConfig = {
|
||||
deployMode,
|
||||
stateBackend,
|
||||
@@ -505,6 +545,7 @@ export function loadGatewayConfig(overrides?: Partial<GatewayConfig>): GatewayCo
|
||||
worker,
|
||||
cos,
|
||||
observability,
|
||||
offload,
|
||||
};
|
||||
|
||||
// Merge overrides one level deep so partial `server`/`data`/`llm` patches
|
||||
|
||||
@@ -46,6 +46,7 @@ export interface ClassifiedError {
|
||||
*
|
||||
* Recognized:
|
||||
* - PayloadTooLargeError (CR-7) — detected via duck-typing on statusCode === 413
|
||||
* - Invalid JSON body — parseJsonBody throws Error("Invalid JSON body") on malformed input
|
||||
* - RecallFailure (H-15) — code from RecallError taxonomy
|
||||
* - SeedValidationError — 400 with generic message
|
||||
* - Anything else — 500 Internal server error
|
||||
@@ -64,6 +65,37 @@ export function classifyError(err: unknown): ClassifiedError {
|
||||
};
|
||||
}
|
||||
|
||||
// 1b. Invalid JSON body / decompression error — parseJsonBody throws this fixed message on
|
||||
// JSON.parse failure or gzip/deflate decompression failure. Client error, not server fault.
|
||||
if (err instanceof Error && err.message === "Invalid JSON body") {
|
||||
return {
|
||||
status: 400,
|
||||
client: { code: 400, message: "Invalid JSON body", trace_id, retryable: false },
|
||||
logLine: `[${trace_id}] InvalidJsonBody: request body could not be parsed as JSON`,
|
||||
};
|
||||
}
|
||||
|
||||
// 1c. COS AppendPositionErr — concurrent append conflict. The client should retry.
|
||||
// This is a transient conflict, not a permanent error.
|
||||
if (err instanceof Error && /AppendPositionErr|Position not equal object length/i.test(err.message)) {
|
||||
return {
|
||||
status: 409,
|
||||
client: { code: 409, message: "Concurrent write conflict, please retry", trace_id, retryable: true },
|
||||
logLine: `[${trace_id}] CosAppendConflict: ${err.message}`,
|
||||
};
|
||||
}
|
||||
|
||||
// 1d. Unsupported Content-Encoding — parseJsonBody rejects with this message when the
|
||||
// client sends an encoding the server does not support (not gzip/deflate/identity).
|
||||
if (err instanceof Error && err.message.startsWith("Unsupported Content-Encoding:")) {
|
||||
const safeMsg = err.message.replace(/[^\w\s:/-]/g, "");
|
||||
return {
|
||||
status: 415,
|
||||
client: { code: 415, message: safeMsg, trace_id, retryable: false },
|
||||
logLine: `[${trace_id}] UnsupportedEncoding: ${err.message}`,
|
||||
};
|
||||
}
|
||||
|
||||
// 2. RecallFailure (H-15) — the recall layer has already produced a safe message.
|
||||
// (Gateway handlers normally translate this into RecallResult.error before reaching
|
||||
// the global catch, but if a code path forgets, this fallback ensures no raw leak.)
|
||||
|
||||
+201
-15
@@ -17,6 +17,7 @@
|
||||
import http from "node:http";
|
||||
import { URL } from "node:url";
|
||||
import { timingSafeEqual } from "node:crypto";
|
||||
import zlib from "node:zlib";
|
||||
import dayjs from "dayjs";
|
||||
import { TdaiCore } from "../core/tdai-core.js";
|
||||
import { StandaloneHostAdapter } from "../adapters/standalone/host-adapter.js";
|
||||
@@ -53,6 +54,9 @@ import { executeSeed } from "../core/seed/seed-runtime.js";
|
||||
import type { SeedProgress } from "../core/seed/types.js";
|
||||
import { handleV2Route, errorEnvelope, makeRequestId } from "./v2-router.js";
|
||||
import type { V2RouterDeps } from "./v2-router.js";
|
||||
import { handleOffloadV2Route } from "../offload_server/router.js";
|
||||
import type { OffloadV2Deps } from "../offload_server/router.js";
|
||||
import { initServerOpikTracer } from "../offload_server/opik-tracer.js";
|
||||
import { classifyError } from "./error-handler.js";
|
||||
import { LocalStorageBackend } from "../core/storage/local-backend.js";
|
||||
import { StorageAdapter } from "../core/storage/adapter.js";
|
||||
@@ -142,11 +146,25 @@ export async function parseJsonBody<T>(req: http.IncomingMessage): Promise<T> {
|
||||
return;
|
||||
}
|
||||
|
||||
// Determine if the body is compressed (Content-Encoding header).
|
||||
// Support gzip and deflate; reject unsupported encodings with 400.
|
||||
const encoding = (req.headers["content-encoding"] ?? "").toLowerCase().trim();
|
||||
let source: NodeJS.ReadableStream = req;
|
||||
if (encoding === "gzip" || encoding === "x-gzip") {
|
||||
source = req.pipe(zlib.createGunzip());
|
||||
} else if (encoding === "deflate") {
|
||||
source = req.pipe(zlib.createInflate());
|
||||
} else if (encoding !== "" && encoding !== "identity") {
|
||||
req.resume(); // drain
|
||||
reject(new Error(`Unsupported Content-Encoding: ${encoding}`));
|
||||
return;
|
||||
}
|
||||
|
||||
const chunks: Buffer[] = [];
|
||||
let received = 0;
|
||||
let aborted = false;
|
||||
|
||||
req.on("data", (chunk: Buffer) => {
|
||||
source.on("data", (chunk: Buffer) => {
|
||||
if (aborted) return;
|
||||
received += chunk.length;
|
||||
if (received > MAX_BODY_BYTES) {
|
||||
@@ -157,18 +175,19 @@ export async function parseJsonBody<T>(req: http.IncomingMessage): Promise<T> {
|
||||
}
|
||||
chunks.push(chunk);
|
||||
});
|
||||
req.on("end", () => {
|
||||
source.on("end", () => {
|
||||
if (aborted) return;
|
||||
try {
|
||||
const body = Buffer.concat(chunks).toString("utf-8");
|
||||
resolve(JSON.parse(body) as T);
|
||||
} catch (err) {
|
||||
} catch {
|
||||
reject(new Error("Invalid JSON body"));
|
||||
}
|
||||
});
|
||||
req.on("error", (err) => {
|
||||
source.on("error", (_err) => {
|
||||
if (aborted) return; // already rejected with PayloadTooLargeError
|
||||
reject(err);
|
||||
// Decompression errors (e.g. truncated gzip) are client-side faults
|
||||
reject(new Error("Invalid JSON body"));
|
||||
});
|
||||
});
|
||||
}
|
||||
@@ -333,6 +352,9 @@ export class TdaiGateway {
|
||||
// Initialize core
|
||||
await this.core.initialize();
|
||||
|
||||
// ── Initialize Opik tracer for offload server ──
|
||||
await initServerOpikTracer(this.logger);
|
||||
|
||||
// ── Initialize StorageAdapter for v2 API ──
|
||||
// In standalone mode, use LocalStorageBackend pointing to dataDir.
|
||||
// In service mode, CosStorageBackend is injected externally.
|
||||
@@ -578,6 +600,17 @@ export class TdaiGateway {
|
||||
}
|
||||
}
|
||||
|
||||
// ── Offload V2 routes (async ingest + mmd query) ──
|
||||
const offloadDeps: OffloadV2Deps = {
|
||||
resolveStorage: v2Deps.resolveStorage,
|
||||
getStorage: v2Deps.getStorage ?? (() => undefined),
|
||||
logger: this.logger,
|
||||
stateBackend: this.stateBackend,
|
||||
config: { ...this.config.offload, l1Model: "", l15Model: "", l2Model: "" },
|
||||
};
|
||||
const offloadHandled = await handleOffloadV2Route(req, res, pathname, method, parseJsonBody, sendJson, offloadDeps);
|
||||
if (offloadHandled) return;
|
||||
|
||||
const handled = await handleV2Route(req, res, pathname, method, parseJsonBody, sendJson, v2Deps);
|
||||
if (handled) return;
|
||||
|
||||
@@ -1101,26 +1134,65 @@ export class TdaiGateway {
|
||||
type: backendType,
|
||||
local: backendType === "local" ? {
|
||||
onTimerExpired: (entry) => {
|
||||
// Parse timer member: "sessionId:L2_schedule" or "sessionId:L1_idle"
|
||||
const parts = entry.member.split(":");
|
||||
const sessionId = parts.slice(0, -1).join(":");
|
||||
const timerType = parts[parts.length - 1];
|
||||
const taskType = timerType === "L2_schedule" ? "L2" : timerType === "L1_idle" ? "L1" : "L3";
|
||||
const instanceId = this.config.instanceId ?? "default";
|
||||
// Parse timer member by prefix: "offload-{type}:{instanceId}:{sessionId}[:{extra}]"
|
||||
// or legacy "session:L2_schedule"
|
||||
const member = entry.member;
|
||||
let taskType: string;
|
||||
let instanceId: string;
|
||||
let sessionId: string;
|
||||
|
||||
const firstColon = member.indexOf(":");
|
||||
const prefix = firstColon > 0 ? member.slice(0, firstColon) : member;
|
||||
|
||||
if (prefix === "offload-l1" || prefix === "offload-l15" || prefix === "offload-l2") {
|
||||
taskType = prefix;
|
||||
// Format: "offload-{type}:{instanceId}:{sessionId}[:{mmdFile}]"
|
||||
// instanceId is the segment right after the prefix
|
||||
const rest = member.slice(firstColon + 1);
|
||||
const instanceEnd = rest.indexOf(":");
|
||||
if (instanceEnd > 0) {
|
||||
instanceId = rest.slice(0, instanceEnd);
|
||||
sessionId = rest.slice(instanceEnd + 1);
|
||||
} else {
|
||||
instanceId = this.config.instanceId ?? "default";
|
||||
sessionId = rest;
|
||||
}
|
||||
// For offload-l2: strip trailing ":{mmdFile}" from sessionId
|
||||
// (mmdFile is extracted separately from timerMember in the executor)
|
||||
if (prefix === "offload-l2" && sessionId.endsWith(".mmd")) {
|
||||
const lastColon = sessionId.lastIndexOf(":");
|
||||
if (lastColon > 0) {
|
||||
sessionId = sessionId.slice(0, lastColon);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// Legacy format: "sessionId:L2_schedule" or "sessionId:L1_idle"
|
||||
const lastColon = member.lastIndexOf(":");
|
||||
const suffix = lastColon >= 0 ? member.slice(lastColon + 1) : "";
|
||||
sessionId = lastColon >= 0 ? member.slice(0, lastColon) : member;
|
||||
taskType = suffix === "L2_schedule" ? "offload-l2" : suffix === "L1_idle" ? "offload-l1" : "L3";
|
||||
instanceId = this.config.instanceId ?? "default";
|
||||
}
|
||||
const now = Date.now();
|
||||
// Extract targetMmdFile from member for offload-l2 (needed by pipeline-worker lockKey)
|
||||
let targetMmdFile: string | undefined;
|
||||
if (taskType === "offload-l2") {
|
||||
const mmdMatch = member.match(/(\d+-[^:]+\.mmd)$/);
|
||||
if (mmdMatch) targetMmdFile = mmdMatch[1];
|
||||
}
|
||||
const task = {
|
||||
id: `${taskType}-${sessionId}-${now}`,
|
||||
type: taskType,
|
||||
type: taskType as any,
|
||||
instanceId,
|
||||
sessionId,
|
||||
priority: 0,
|
||||
createdAt: now,
|
||||
data: { triggeredBy: "timer_scanner", timerMember: entry.member, instanceId },
|
||||
data: { triggeredBy: "timer_scanner", timerMember: member, instanceId, targetMmdFile },
|
||||
};
|
||||
this.stateBackend!.enqueueTask(task).then(() => {
|
||||
this.logger.info(`[local-timer] Timer fired: ${entry.member} → enqueued ${taskType} task`);
|
||||
this.logger.info(`[local-timer] Timer fired: ${member} → enqueued ${taskType} task`);
|
||||
}).catch((err) => {
|
||||
this.logger.error(`[local-timer] Failed to enqueue task for ${entry.member}: ${err instanceof Error ? err.message : String(err)}`);
|
||||
this.logger.error(`[local-timer] Failed to enqueue task for ${member}: ${err instanceof Error ? err.message : String(err)}`);
|
||||
});
|
||||
},
|
||||
} : undefined,
|
||||
@@ -1571,6 +1643,120 @@ export class TdaiGateway {
|
||||
async executeFlush(task: TaskPayload) {
|
||||
await core.handleSessionEnd(task.sessionId);
|
||||
},
|
||||
|
||||
// ── Offload executors (L1 summary, L1.5 task judgment, L2 MMD update) ──
|
||||
async executeOffloadL1(task: TaskPayload, signal?: AbortSignal) {
|
||||
if (signal?.aborted) return;
|
||||
const { OffloadTaskExecutor } = await import("../offload_server/offload-task-executor.js");
|
||||
const storage = await resolveStorage(task);
|
||||
if (!storage) return;
|
||||
const llmClient = gateway.buildOffloadLlmClient();
|
||||
if (!llmClient) {
|
||||
gateway.logger.warn(`[executor] offload-l1 skipped: no LLM client available`);
|
||||
return;
|
||||
}
|
||||
const executor = new OffloadTaskExecutor({
|
||||
resolveStorage: async () => storage,
|
||||
llmClient,
|
||||
stateBackend: gateway.stateBackend!,
|
||||
config: { ...gateway.config.offload, l1Model: "", l15Model: "", l2Model: "" },
|
||||
logger: gateway.logger,
|
||||
});
|
||||
await executor.executeOffloadL1(task, signal);
|
||||
},
|
||||
async executeOffloadL15(task: TaskPayload, signal?: AbortSignal) {
|
||||
if (signal?.aborted) return;
|
||||
const { OffloadTaskExecutor } = await import("../offload_server/offload-task-executor.js");
|
||||
const storage = await resolveStorage(task);
|
||||
if (!storage) return;
|
||||
const llmClient = gateway.buildOffloadLlmClient();
|
||||
if (!llmClient) {
|
||||
gateway.logger.warn(`[executor] offload-l15 skipped: no LLM client available`);
|
||||
return;
|
||||
}
|
||||
const executor = new OffloadTaskExecutor({
|
||||
resolveStorage: async () => storage,
|
||||
llmClient,
|
||||
stateBackend: gateway.stateBackend!,
|
||||
config: { ...gateway.config.offload, l1Model: "", l15Model: "", l2Model: "" },
|
||||
logger: gateway.logger,
|
||||
});
|
||||
await executor.executeOffloadL15(task, signal);
|
||||
},
|
||||
async executeOffloadL2(task: TaskPayload, signal?: AbortSignal) {
|
||||
if (signal?.aborted) return;
|
||||
const { OffloadTaskExecutor } = await import("../offload_server/offload-task-executor.js");
|
||||
const storage = await resolveStorage(task);
|
||||
if (!storage) return;
|
||||
const llmClient = gateway.buildOffloadLlmClient();
|
||||
if (!llmClient) {
|
||||
gateway.logger.warn(`[executor] offload-l2 skipped: no LLM client available`);
|
||||
return;
|
||||
}
|
||||
const executor = new OffloadTaskExecutor({
|
||||
resolveStorage: async () => storage,
|
||||
llmClient,
|
||||
stateBackend: gateway.stateBackend!,
|
||||
config: { ...gateway.config.offload, l1Model: "", l15Model: "", l2Model: "" },
|
||||
logger: gateway.logger,
|
||||
});
|
||||
await executor.executeOffloadL2(task, signal);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a simple LLM client for offload executors using gateway's LLM config.
|
||||
*/
|
||||
private buildOffloadLlmClient() {
|
||||
const llmCfg = this.config.llm;
|
||||
if (!llmCfg.baseUrl || !llmCfg.apiKey || !llmCfg.model) return null;
|
||||
const logger = this.logger;
|
||||
|
||||
return {
|
||||
async chat(params: {
|
||||
model: string;
|
||||
messages: Array<{ role: "system" | "user"; content: string }>;
|
||||
temperature: number;
|
||||
max_tokens: number;
|
||||
timeoutMs?: number;
|
||||
}): Promise<string> {
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), params.timeoutMs ?? 30000);
|
||||
try {
|
||||
const response = await fetch(`${llmCfg.baseUrl}/chat/completions`, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
Authorization: `Bearer ${llmCfg.apiKey}`,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: llmCfg.model || params.model,
|
||||
messages: params.messages,
|
||||
temperature: params.temperature,
|
||||
max_tokens: params.max_tokens,
|
||||
}),
|
||||
signal: controller.signal,
|
||||
});
|
||||
clearTimeout(timer);
|
||||
if (!response.ok) {
|
||||
throw new Error(`LLM API returned ${response.status}: ${await response.text()}`);
|
||||
}
|
||||
const json = (await response.json()) as any;
|
||||
const finishReason = json.choices?.[0]?.finish_reason;
|
||||
if (finishReason === "length") {
|
||||
const content = json.choices?.[0]?.message?.content ?? "";
|
||||
logger.warn(
|
||||
`[offload-llm] Response truncated (finish_reason=length, max_tokens=${params.max_tokens}), ` +
|
||||
`content=${content.length} chars`,
|
||||
);
|
||||
}
|
||||
return json.choices?.[0]?.message?.content ?? "";
|
||||
} catch (err) {
|
||||
clearTimeout(timer);
|
||||
throw err;
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,526 @@
|
||||
/**
|
||||
* offload-client — OffloadContextEngine.
|
||||
* Occupies the Context Engine slot and delegates compression to the server.
|
||||
*/
|
||||
import type { OffloadClientConfig, RecentMessage, Logger } from "./types.js";
|
||||
import type { OffloadApiClient } from "./offload-api-client.js";
|
||||
import { estimateAllTokens, estimateMessageTokens } from "./token-estimator.js";
|
||||
|
||||
const DEFAULT_CONTEXT_WINDOW = 128000;
|
||||
/** compact target: keep messages until total <= contextWindow * TARGET_RATIO */
|
||||
const COMPACT_TARGET_RATIO = 0.5;
|
||||
/** When truncating a single large tool_result, keep at most this many chars */
|
||||
const TOOL_RESULT_TRUNCATE_CHARS = 2000;
|
||||
|
||||
// ─── Message role helpers (handle multiple formats) ─────────────────────────
|
||||
|
||||
function getMsgRole(msg: any): string {
|
||||
return msg?.role ?? msg?.message?.role ?? msg?.type ?? "";
|
||||
}
|
||||
|
||||
function isToolResult(msg: any): boolean {
|
||||
const role = getMsgRole(msg);
|
||||
if (role === "tool" || role === "toolResult" || role === "tool_result") return true;
|
||||
// Anthropic: user message with tool_result content blocks
|
||||
if (role === "user" && Array.isArray(msg?.content)) {
|
||||
return msg.content.some((b: any) => b?.type === "tool_result");
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
function isAssistantWithToolUse(msg: any): boolean {
|
||||
const role = getMsgRole(msg);
|
||||
if (role !== "assistant") return false;
|
||||
const content = msg?.type === "message" ? msg?.message?.content : msg?.content;
|
||||
if (!Array.isArray(content)) return false;
|
||||
return content.some((b: any) => b?.type === "tool_use" || b?.type === "toolCall");
|
||||
}
|
||||
|
||||
/**
|
||||
* Truncate tool_result content in-place, returning a shallow-cloned message.
|
||||
*/
|
||||
function truncateToolResult(msg: any, maxChars: number): any {
|
||||
const clone = JSON.parse(JSON.stringify(msg));
|
||||
const content = clone.type === "message" ? clone.message?.content : clone.content;
|
||||
if (typeof content === "string" && content.length > maxChars) {
|
||||
const truncated = content.slice(0, maxChars) + "\n...[truncated]";
|
||||
if (clone.type === "message") clone.message.content = truncated;
|
||||
else clone.content = truncated;
|
||||
return clone;
|
||||
}
|
||||
if (Array.isArray(content)) {
|
||||
for (const block of content) {
|
||||
if (typeof block?.text === "string" && block.text.length > maxChars) {
|
||||
block.text = block.text.slice(0, maxChars) + "\n...[truncated]";
|
||||
}
|
||||
if (typeof block?.content === "string" && block.content.length > maxChars) {
|
||||
block.content = block.content.slice(0, maxChars) + "\n...[truncated]";
|
||||
}
|
||||
}
|
||||
return clone;
|
||||
}
|
||||
return clone;
|
||||
}
|
||||
|
||||
export class OffloadContextEngine {
|
||||
/** Per-session state cache. */
|
||||
private sessions = new Map<string, {
|
||||
lastAccessMs: number; // ← NEW: track last access time for cleanup
|
||||
lastKnownTotalTokens: number;
|
||||
lastKnownMsgCount: number;
|
||||
lastL15PromptHash: string;
|
||||
cachedPrompt?: string;
|
||||
cachedRecentMessages: RecentMessage[];
|
||||
cachedRecentContext?: string;
|
||||
}>();
|
||||
|
||||
/** Get or create per-session state. */
|
||||
private getSession(sessionKey: string) {
|
||||
let s = this.sessions.get(sessionKey);
|
||||
if (!s) {
|
||||
s = {
|
||||
lastAccessMs: Date.now(), // ← NEW
|
||||
lastKnownTotalTokens: 0,
|
||||
lastKnownMsgCount: 0,
|
||||
lastL15PromptHash: "",
|
||||
cachedRecentMessages: [],
|
||||
};
|
||||
this.sessions.set(sessionKey, s);
|
||||
} else {
|
||||
s.lastAccessMs = Date.now(); // ← NEW: update on access
|
||||
}
|
||||
return s;
|
||||
}
|
||||
|
||||
/**
|
||||
* Reset session state (call when session is /new'd or destroyed).
|
||||
*/
|
||||
resetSession(sessionKey: string): void {
|
||||
this.sessions.delete(sessionKey);
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear all session states (emergency shutdown).
|
||||
*/
|
||||
clearAllSessions(): void {
|
||||
const n = this.sessions.size;
|
||||
this.sessions.clear();
|
||||
if (n > 0) {
|
||||
this.logger.info(`[offload-client] cleared ${n} session states`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the cached context for a session (for after_tool_call hook to send with ingest).
|
||||
*/
|
||||
getContext(sessionKey?: string): { prompt?: string; recentMessages?: RecentMessage[] } | undefined {
|
||||
if (!sessionKey) return undefined;
|
||||
const s = this.sessions.get(sessionKey);
|
||||
if (!s || (!s.cachedPrompt && s.cachedRecentMessages.length === 0)) return undefined;
|
||||
return { prompt: s.cachedPrompt, recentMessages: s.cachedRecentMessages };
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the cached formatted context string for a session (for legacy compatibility).
|
||||
*/
|
||||
getRecentContext(sessionKey?: string): string | undefined {
|
||||
if (!sessionKey) return undefined;
|
||||
return this.sessions.get(sessionKey)?.cachedRecentContext;
|
||||
}
|
||||
|
||||
constructor(
|
||||
private client: OffloadApiClient,
|
||||
private config: OffloadClientConfig,
|
||||
private logger: Logger,
|
||||
) {}
|
||||
|
||||
get info() {
|
||||
return {
|
||||
id: "memory-tencentdb",
|
||||
name: "Offload Client Context Engine",
|
||||
version: "2.0.0",
|
||||
ownsCompaction: true,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* bootstrap — called when a new session starts (e.g. /new command).
|
||||
* Resets per-session cached state.
|
||||
*/
|
||||
async bootstrap(params: { sessionKey?: string; sessionId?: string }) {
|
||||
const sk = params.sessionKey ?? params.sessionId;
|
||||
if (sk) this.resetSession(sk);
|
||||
return { bootstrapped: true };
|
||||
}
|
||||
|
||||
/**
|
||||
* ingest — no-op for client mode (ingest is handled by after_tool_call hook).
|
||||
* Required by the framework ContextEngine interface.
|
||||
*/
|
||||
async ingest(_params: {
|
||||
sessionId: string;
|
||||
sessionKey?: string;
|
||||
message: any;
|
||||
isHeartbeat?: boolean;
|
||||
}) {
|
||||
return { ingested: true };
|
||||
}
|
||||
|
||||
/**
|
||||
* compact — record the framework's authoritative token count for calibration,
|
||||
* then defer actual compaction to assemble().
|
||||
*/
|
||||
async compact(params: {
|
||||
sessionId: string;
|
||||
sessionKey?: string;
|
||||
sessionFile: string;
|
||||
tokenBudget?: number;
|
||||
force?: boolean;
|
||||
currentTokenCount?: number;
|
||||
compactionTarget?: "budget" | "threshold";
|
||||
customInstructions?: string;
|
||||
runtimeContext?: any;
|
||||
}) {
|
||||
// Record framework token count for calibration
|
||||
if (params.currentTokenCount && params.currentTokenCount > 0) {
|
||||
const sk = params.sessionKey ?? params.sessionId;
|
||||
const s = this.getSession(sk);
|
||||
s.lastKnownTotalTokens = params.currentTokenCount;
|
||||
this.logger.debug?.(
|
||||
`[offload-client] compact: calibration updated, knownTokens=${s.lastKnownTotalTokens}`,
|
||||
);
|
||||
}
|
||||
|
||||
const contextWindow = params.tokenBudget ?? DEFAULT_CONTEXT_WINDOW;
|
||||
const targetTokens = Math.floor(contextWindow * COMPACT_TARGET_RATIO);
|
||||
|
||||
this.logger.info(
|
||||
`[offload-client] compact: sessionKey=${params.sessionKey ?? params.sessionId}, ` +
|
||||
`budget=${contextWindow}, target=${targetTokens}, ` +
|
||||
`currentTokens=${params.currentTokenCount}, force=${params.force}`,
|
||||
);
|
||||
|
||||
return {
|
||||
ok: true,
|
||||
compacted: false,
|
||||
reason: "offload-client: compaction deferred to assemble()",
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the best known token total for calibration.
|
||||
* Uses lastKnownTotalTokens only if message count hasn't changed significantly.
|
||||
*/
|
||||
private resolveCalibrationTokens(sessionKey: string, msgCount: number): number | undefined {
|
||||
const s = this.sessions.get(sessionKey);
|
||||
if (!s || s.lastKnownTotalTokens <= 0) return undefined;
|
||||
// If message count changed by >20%, the cached total is stale — skip calibration
|
||||
if (s.lastKnownMsgCount > 0 && Math.abs(msgCount - s.lastKnownMsgCount) / s.lastKnownMsgCount > 0.2) {
|
||||
return undefined;
|
||||
}
|
||||
return s.lastKnownTotalTokens;
|
||||
}
|
||||
|
||||
/**
|
||||
* Local brute-force compaction: keep tail messages up to target budget,
|
||||
* respecting tool pairs and truncating oversized tool_results.
|
||||
* Used as fallback when server compaction is unavailable.
|
||||
*/
|
||||
private localCompact(messages: any[], contextWindow: number, sessionKey: string): any[] {
|
||||
const targetTokens = Math.floor(contextWindow * COMPACT_TARGET_RATIO);
|
||||
const knownTokens = this.resolveCalibrationTokens(sessionKey, messages.length);
|
||||
const { perMessage } = estimateAllTokens(messages, knownTokens);
|
||||
const n = messages.length;
|
||||
|
||||
// Step 1: scan from tail, find the cut index
|
||||
let cumTokens = 0;
|
||||
let cutIdx = n; // everything before cutIdx gets deleted
|
||||
for (let i = n - 1; i >= 0; i--) {
|
||||
cumTokens += perMessage[i];
|
||||
if (cumTokens > targetTokens) {
|
||||
cutIdx = i + 1;
|
||||
break;
|
||||
}
|
||||
cutIdx = i;
|
||||
}
|
||||
|
||||
// Never delete the very first user message (index 0)
|
||||
if (cutIdx <= 0) cutIdx = 0;
|
||||
|
||||
// Step 2: expand cut boundary to respect tool pairs
|
||||
// If cutIdx lands inside a tool pair, move it to include the full pair.
|
||||
|
||||
// 2a: If msg at cutIdx is a tool_result, its paired assistant+tool_use is
|
||||
// before cutIdx (would be deleted). Move cutIdx back to include the pair.
|
||||
while (cutIdx < n && isToolResult(messages[cutIdx])) {
|
||||
cutIdx++;
|
||||
}
|
||||
|
||||
// 2b: If msg at cutIdx-1 (last deleted) is assistant+tool_use, its tool_result
|
||||
// at cutIdx would be orphaned. Pull cutIdx back to keep the pair.
|
||||
while (cutIdx > 0 && cutIdx < n && isAssistantWithToolUse(messages[cutIdx - 1])) {
|
||||
cutIdx--;
|
||||
}
|
||||
|
||||
// Step 3: build retained array
|
||||
const retained = messages.slice(cutIdx);
|
||||
|
||||
if (retained.length === 0) {
|
||||
return [...messages]; // safety: don't delete everything
|
||||
}
|
||||
|
||||
const deletedCount = cutIdx;
|
||||
let retainedTokens = 0;
|
||||
for (let i = cutIdx; i < n; i++) retainedTokens += perMessage[i];
|
||||
|
||||
this.logger.info(
|
||||
`[offload-client] localCompact: deleted ${deletedCount}/${n} msgs, ` +
|
||||
`retained ${retained.length} msgs, tokens=${retainedTokens}/${targetTokens} target`,
|
||||
);
|
||||
|
||||
// Step 4: if still over target and there's a large tool_result, truncate it
|
||||
if (retainedTokens > targetTokens) {
|
||||
let maxTrIdx = -1;
|
||||
let maxTrTokens = 0;
|
||||
for (let i = 0; i < retained.length; i++) {
|
||||
if (isToolResult(retained[i])) {
|
||||
const t = estimateMessageTokens(retained[i]);
|
||||
if (t > maxTrTokens) {
|
||||
maxTrTokens = t;
|
||||
maxTrIdx = i;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (maxTrIdx >= 0 && maxTrTokens > TOOL_RESULT_TRUNCATE_CHARS / 4) {
|
||||
retained[maxTrIdx] = truncateToolResult(retained[maxTrIdx], TOOL_RESULT_TRUNCATE_CHARS);
|
||||
const newTokens = estimateMessageTokens(retained[maxTrIdx]);
|
||||
retainedTokens = retainedTokens - maxTrTokens + newTokens;
|
||||
this.logger.info(
|
||||
`[offload-client] localCompact: truncated tool_result[${maxTrIdx}] ` +
|
||||
`${maxTrTokens}→${newTokens} tokens, total now=${retainedTokens}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
return retained;
|
||||
}
|
||||
|
||||
// ─── L1.5 Trigger ───────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Trigger L1.5 task judgment via ingest API when a new user prompt is detected.
|
||||
* Fire-and-forget — does not block the assemble flow.
|
||||
*/
|
||||
private triggerL15IfNeeded(prompt: string | undefined, messages: any[], sessionKey: string): void {
|
||||
if (!prompt || typeof prompt !== "string" || prompt.length === 0) return;
|
||||
|
||||
// Skip system/internal prompts that are not user-initiated
|
||||
if (this.isInternalPrompt(prompt)) {
|
||||
this.logger.debug?.(`[offload-client] L1.5 skipped: internal prompt (${prompt.slice(0, 60)})`);
|
||||
return;
|
||||
}
|
||||
|
||||
// Always update cached context for after_tool_call hook (L1 needs it)
|
||||
const recentMsgs = this.buildRecentMessages(prompt, messages);
|
||||
const s = this.getSession(sessionKey);
|
||||
s.cachedPrompt = prompt.slice(0, 500);
|
||||
s.cachedRecentMessages = recentMsgs;
|
||||
s.cachedRecentContext = this.formatContextForL1(prompt, recentMsgs);
|
||||
|
||||
// Dedup: skip L1.5 if same prompt as last trigger for this session
|
||||
const hash = this.simpleHash(prompt);
|
||||
if (s.lastL15PromptHash === hash) {
|
||||
this.logger.debug?.(`[offload-client] L1.5 skipped: same prompt hash (${hash})`);
|
||||
return;
|
||||
}
|
||||
s.lastL15PromptHash = hash;
|
||||
|
||||
this.logger.info(
|
||||
`[offload-client] L1.5 triggered: promptHash=${hash}, recentMsgs=${recentMsgs.length}`,
|
||||
);
|
||||
|
||||
// Fire-and-forget L1.5
|
||||
this.client.ingestL15(sessionKey, prompt.slice(0, 500), recentMsgs).catch((err) => {
|
||||
this.logger.warn(`[offload-client] L1.5 ingestL15 failed: ${err}`);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Detect internal/system prompts that should not trigger L1.5.
|
||||
* These are framework-generated messages, not user-initiated conversations.
|
||||
*/
|
||||
private isInternalPrompt(prompt: string): boolean {
|
||||
// Compaction flush prompts
|
||||
if (prompt.startsWith("Pre-compaction")) return true;
|
||||
// Inter-session routing messages
|
||||
if (prompt.startsWith("[Inter-session message]")) return true;
|
||||
// Heartbeat/keepalive
|
||||
if (prompt.includes("HEARTBEAT") || prompt.includes("heartbeat")) return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build structured RecentMessage[] for ingest API.
|
||||
* Filters: user/assistant text only, no tool calls, no heartbeats.
|
||||
* Max 5 recent turns, 400 chars per message.
|
||||
*/
|
||||
private buildRecentMessages(prompt: string, messages: any[]): RecentMessage[] {
|
||||
const normalizedPrompt = prompt.trim().slice(0, 200).toLowerCase();
|
||||
|
||||
// Scan messages, collect user/assistant pairs
|
||||
const pairs: RecentMessage[] = [];
|
||||
for (const msg of messages) {
|
||||
const role = getMsgRole(msg);
|
||||
|
||||
// Skip tool messages entirely
|
||||
if (isToolResult(msg) || isAssistantWithToolUse(msg)) continue;
|
||||
if (role === "tool" || role === "toolResult" || role === "tool_result") continue;
|
||||
|
||||
if (role === "user") {
|
||||
const text = this.extractMsgText(msg);
|
||||
if (!text || text.length <= 5) continue;
|
||||
if (text.includes("HEARTBEAT") || text.includes("heartbeat")) continue;
|
||||
const trimmed = text.slice(0, 400);
|
||||
// Skip if it matches current prompt
|
||||
const normalizedText = trimmed.slice(0, 200).toLowerCase();
|
||||
if (normalizedPrompt && (normalizedText === normalizedPrompt || normalizedText.startsWith(normalizedPrompt) || normalizedPrompt.startsWith(normalizedText))) continue;
|
||||
pairs.push({ role: "user", content: trimmed });
|
||||
} else if (role === "assistant") {
|
||||
const text = this.extractMsgText(msg);
|
||||
if (!text || text.length <= 10) continue;
|
||||
if (text.includes("HEARTBEAT") || text.includes("heartbeat")) continue;
|
||||
pairs.push({ role: "assistant", content: text.slice(0, 400) });
|
||||
}
|
||||
}
|
||||
|
||||
// Keep last N messages (max 10 messages ≈ 5 turns)
|
||||
const recent = pairs.slice(-10);
|
||||
return recent;
|
||||
}
|
||||
|
||||
/**
|
||||
* Format context string for L1 executor (recent-context.txt).
|
||||
*/
|
||||
private formatContextForL1(prompt: string, recentMsgs: RecentMessage[]): string {
|
||||
const parts: string[] = [];
|
||||
if (recentMsgs.length > 0) {
|
||||
parts.push("历史消息,可作为参考:");
|
||||
for (const m of recentMsgs) {
|
||||
parts.push(`[${m.role === "user" ? "User" : "Assistant"}]: ${m.content}`);
|
||||
}
|
||||
}
|
||||
parts.push(`\n最新user message:\n[User]: ${prompt.slice(0, 500)}`);
|
||||
return parts.join("\n");
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract text content from a message.
|
||||
*/
|
||||
private extractMsgText(msg: any): string {
|
||||
const content = msg?.content ?? msg?.message?.content ?? "";
|
||||
if (typeof content === "string") return content;
|
||||
if (Array.isArray(content)) {
|
||||
return content
|
||||
.map((b: any) => (typeof b === "string" ? b : b?.text ?? ""))
|
||||
.join("");
|
||||
}
|
||||
return "";
|
||||
}
|
||||
|
||||
/**
|
||||
* Simple string hash for prompt deduplication.
|
||||
*/
|
||||
private simpleHash(str: string): string {
|
||||
let hash = 0;
|
||||
for (let i = 0; i < str.length; i++) {
|
||||
hash = ((hash << 5) - hash + str.charCodeAt(i)) | 0;
|
||||
}
|
||||
return hash.toString(36);
|
||||
}
|
||||
|
||||
/**
|
||||
* assemble — estimate ratio → call server compaction → fallback to localCompact.
|
||||
* Framework calls this to build the model context for each turn.
|
||||
*/
|
||||
async assemble(params: {
|
||||
sessionId: string;
|
||||
sessionKey?: string;
|
||||
messages?: any[];
|
||||
tokenBudget?: number;
|
||||
prompt?: string;
|
||||
availableTools?: Set<string>;
|
||||
citationsMode?: string;
|
||||
model?: string;
|
||||
}) {
|
||||
const { messages, sessionKey, sessionId } = params;
|
||||
if (!messages || messages.length === 0) {
|
||||
return { messages: messages ? [...messages] : [], estimatedTokens: 0 };
|
||||
}
|
||||
|
||||
const sk = sessionKey ?? sessionId ?? "unknown";
|
||||
|
||||
// ── L1.5 trigger: fire-and-forget on new user prompt ──
|
||||
this.triggerL15IfNeeded(params.prompt, messages, sk);
|
||||
|
||||
const contextWindow = params.tokenBudget ?? DEFAULT_CONTEXT_WINDOW;
|
||||
// Don't use framework's knownTokens for calibration — our tiktoken is already precise.
|
||||
// Framework's currentTokenCount may use a different calculation method (e.g. chars/4).
|
||||
const { total, perMessage } = estimateAllTokens(messages);
|
||||
const ratio = total / contextWindow;
|
||||
|
||||
// Update calibration state for future calls
|
||||
const s = this.getSession(sk);
|
||||
s.lastKnownTotalTokens = total;
|
||||
s.lastKnownMsgCount = messages.length;
|
||||
|
||||
// Below client threshold — skip compaction
|
||||
if (ratio < this.config.compactionRatio) {
|
||||
this.logger.debug?.(
|
||||
`[offload-client] assemble: ratio=${(ratio * 100).toFixed(1)}% < ${(this.config.compactionRatio * 100).toFixed(0)}%, skip`,
|
||||
);
|
||||
return { messages: [...messages], estimatedTokens: total };
|
||||
}
|
||||
|
||||
this.logger.info(
|
||||
`[offload-client] assemble: ratio=${(ratio * 100).toFixed(1)}%, msgs=${messages.length}, calling compaction...`,
|
||||
);
|
||||
|
||||
// Try server-side compaction first
|
||||
const result = await this.client.compaction({
|
||||
sessionId: sessionKey ?? sessionId ?? "unknown",
|
||||
messages,
|
||||
ratio,
|
||||
contextWindow,
|
||||
totalTokens: total,
|
||||
messageTokens: perMessage,
|
||||
});
|
||||
|
||||
if (result) {
|
||||
const compactedTokens = result.messages.reduce(
|
||||
(sum: number, msg: any) => sum + estimateMessageTokens(msg),
|
||||
0,
|
||||
);
|
||||
this.logger.info(
|
||||
`[offload-client] server compaction done: level=${result.report.resolvedLevel}, ` +
|
||||
`${result.report.originalCount}→${result.report.compactedCount} msgs, ` +
|
||||
`mild=${result.report.mildReplacements}, agg=${result.report.aggressiveDeleted}, ` +
|
||||
`em=${result.report.emergencyDeleted}, mmd=${result.report.mmdInjected}`,
|
||||
);
|
||||
return { messages: result.messages, estimatedTokens: compactedTokens };
|
||||
}
|
||||
|
||||
// Fallback: local brute-force compaction
|
||||
this.logger.warn("[offload-client] server compaction failed, falling back to local compact");
|
||||
const compacted = this.localCompact(messages, contextWindow, sk);
|
||||
const compactedTokens = compacted.reduce(
|
||||
(sum: number, msg: any) => sum + estimateMessageTokens(msg),
|
||||
0,
|
||||
);
|
||||
return { messages: compacted, estimatedTokens: compactedTokens };
|
||||
}
|
||||
|
||||
/**
|
||||
* afterTurn — no-op (ingest is handled by after_tool_call hook).
|
||||
*/
|
||||
async afterTurn() {}
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
/**
|
||||
* offload-client — after_tool_call hook handler.
|
||||
* Fire-and-forget: sends tool pair + context to ingest API for L1 processing.
|
||||
*/
|
||||
import type { OffloadApiClient } from "../offload-api-client.js";
|
||||
import type { OffloadClientConfig, ToolPairPayload, RecentMessage, Logger } from "../types.js";
|
||||
|
||||
export interface AfterToolCallEvent {
|
||||
toolName: string;
|
||||
toolCallId: string;
|
||||
params?: unknown;
|
||||
result?: unknown;
|
||||
error?: string;
|
||||
durationMs?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Create the after_tool_call hook handler.
|
||||
* Sends each tool call result to the server for L1 processing.
|
||||
*
|
||||
* @param getContext Optional getter for { prompt, recentMessages } context per session.
|
||||
*/
|
||||
export function createAfterToolCallHandler(
|
||||
client: OffloadApiClient,
|
||||
config: OffloadClientConfig,
|
||||
logger: Logger,
|
||||
getContext?: (sessionKey: string) => { prompt?: string; recentMessages?: RecentMessage[] } | undefined,
|
||||
) {
|
||||
return (event: AfterToolCallEvent, ctx: { sessionKey?: string; sessionId?: string }) => {
|
||||
const sessionId = ctx.sessionKey ?? ctx.sessionId;
|
||||
logger.debug?.(
|
||||
`[offload-client] after_tool_call: tool=${event.toolName}, session=${sessionId ?? "(none)"}, callId=${event.toolCallId ?? "(none)"}`,
|
||||
);
|
||||
if (!sessionId) return;
|
||||
|
||||
const toolPair: ToolPairPayload = {
|
||||
toolName: event.toolName,
|
||||
toolCallId: event.toolCallId,
|
||||
params: event.params ?? {},
|
||||
result: event.result,
|
||||
error: event.error,
|
||||
timestamp: new Date().toISOString(),
|
||||
durationMs: event.durationMs,
|
||||
};
|
||||
|
||||
const context = getContext?.(sessionId);
|
||||
|
||||
// Fire-and-forget — do not block the LLM flow
|
||||
client.ingestWithContext(sessionId, [toolPair], context?.prompt, context?.recentMessages).catch((err) => {
|
||||
logger.warn(`[offload-client] ingest fire-and-forget error: ${err}`);
|
||||
});
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,101 @@
|
||||
/**
|
||||
* offload-client — Plugin registration entry point.
|
||||
* Stateless, server-delegated offload client.
|
||||
* 1 hook (after_tool_call) + 1 Context Engine (assemble → compaction API).
|
||||
*/
|
||||
import type { OffloadClientConfig, Logger } from "./types.js";
|
||||
import { defaultOffloadClientConfig } from "./types.js";
|
||||
import { OffloadApiClient } from "./offload-api-client.js";
|
||||
import { OffloadContextEngine } from "./context-engine.js";
|
||||
import { createAfterToolCallHandler } from "./hooks/after-tool-call.js";
|
||||
|
||||
export interface OpenClawPluginApi {
|
||||
on: (hookName: string, handler: (...args: any[]) => any) => void;
|
||||
registerContextEngine: (id: string, factoryOrInstance: any) => any;
|
||||
logger?: Logger;
|
||||
}
|
||||
|
||||
/**
|
||||
* Register the offload-client plugin.
|
||||
* Call this from the main plugin's register() when offload-client config is enabled.
|
||||
*/
|
||||
export function registerOffloadClient(api: OpenClawPluginApi, userConfig: Partial<OffloadClientConfig>): void {
|
||||
const config: OffloadClientConfig = { ...defaultOffloadClientConfig(), ...userConfig };
|
||||
const logger: Logger = api.logger ?? { info: console.log, warn: console.warn, error: console.error, debug: console.debug };
|
||||
|
||||
if (!config.enabled) {
|
||||
logger.info("[offload-client] disabled by config");
|
||||
return;
|
||||
}
|
||||
|
||||
if (!config.serverUrl || !config.apiKey || !config.serviceId) {
|
||||
logger.error("[offload-client] missing required config: serverUrl, apiKey, or serviceId");
|
||||
return;
|
||||
}
|
||||
|
||||
const client = new OffloadApiClient(config, logger);
|
||||
|
||||
// Context Engine: occupies slot, assemble() calls compaction API
|
||||
const engine = new OffloadContextEngine(client, config, logger);
|
||||
|
||||
// Hook: fire-and-forget ingest on every tool call (with context from engine per session)
|
||||
const afterToolCallHandler = createAfterToolCallHandler(
|
||||
client, config, logger,
|
||||
(sessionKey) => engine.getContext(sessionKey),
|
||||
config.agentName, // ← NEW: pass agentName for sessionId construction
|
||||
);
|
||||
api.on("after_tool_call", afterToolCallHandler);
|
||||
|
||||
// ── Memory management hooks ──
|
||||
|
||||
// agent_end: clear token cache after each agent turn
|
||||
api.on("agent_end", (_event: any, ctx: { sessionKey?: string; sessionId?: string }) => {
|
||||
const sk = ctx.sessionKey ?? ctx.sessionId;
|
||||
if (sk) {
|
||||
engine.resetSession(sk);
|
||||
logger.debug?.(`[offload-client] reset session state: ${sk}`);
|
||||
}
|
||||
});
|
||||
|
||||
// gateway_stop: emergency cleanup on shutdown
|
||||
api.on("gateway_stop", async () => {
|
||||
engine.clearAllSessions();
|
||||
logger.info("[offload-client] all session states cleared on gateway_stop");
|
||||
});
|
||||
|
||||
try {
|
||||
const result = api.registerContextEngine("memory-tencentdb", () => engine) as any;
|
||||
if (result?.ok === false) {
|
||||
logger.error(
|
||||
`[offload-client] Context Engine slot occupied by "${result.existingOwner ?? "unknown"}". ` +
|
||||
`Compaction disabled — only ingest will work.`,
|
||||
);
|
||||
} else {
|
||||
logger.info("[offload-client] Context Engine registered");
|
||||
}
|
||||
} catch (err) {
|
||||
logger.error(`[offload-client] registerContextEngine failed: ${err}. Compaction disabled.`);
|
||||
}
|
||||
|
||||
// ── Health check (async, non-blocking) ──
|
||||
client.checkHealth().then((ok) => {
|
||||
if (!ok) {
|
||||
logger.warn(
|
||||
`[offload-client] ⚠️ Server ${config.serverUrl} unreachable! ` +
|
||||
`Ingest calls will fail silently until server becomes available.`
|
||||
);
|
||||
} else {
|
||||
logger.info(`[offload-client] Server health OK: ${config.serverUrl}`);
|
||||
}
|
||||
}).catch((_err) => {
|
||||
// ignore
|
||||
});
|
||||
|
||||
logger.info(`[offload-client] registered (server=${config.serverUrl})`);
|
||||
}
|
||||
|
||||
export { OffloadApiClient } from "./offload-api-client.js";
|
||||
export { OffloadContextEngine } from "./context-engine.js";
|
||||
export { createAfterToolCallHandler } from "./hooks/after-tool-call.js";
|
||||
export { estimateTokens, estimateMessageTokens, estimateAllTokens } from "./token-estimator.js";
|
||||
export type { OffloadClientConfig, ToolPairPayload, CompactionResult, CompactionReport, Logger } from "./types.js";
|
||||
@@ -0,0 +1,177 @@
|
||||
/**
|
||||
* offload-client — HTTP client for Offload Server v2 API.
|
||||
*/
|
||||
import type { OffloadClientConfig, ToolPairPayload, RecentMessage, CompactionResult, Logger } from "./types.js";
|
||||
|
||||
export class OffloadApiClient {
|
||||
constructor(
|
||||
private config: OffloadClientConfig,
|
||||
private logger: Logger,
|
||||
) {}
|
||||
|
||||
/** Health check: GET /v2/offload/health. Returns true if server is reachable (any HTTP response = reachable). */
|
||||
async checkHealth(): Promise<boolean> {
|
||||
try {
|
||||
const controller = new AbortController();
|
||||
const timeoutId = setTimeout(() => controller.abort(), 5000);
|
||||
const res = await fetch(`${this.config.serverUrl}/v2/offload/health`, {
|
||||
method: "GET",
|
||||
headers: { Authorization: `Bearer ${this.config.apiKey}` },
|
||||
signal: controller.signal,
|
||||
});
|
||||
clearTimeout(timeoutId);
|
||||
// Any HTTP response (including 401/403) means server is reachable
|
||||
return res.status < 500;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Fire-and-forget: send tool pairs to ingest endpoint.
|
||||
* Does not throw — failures are logged as warnings.
|
||||
*/
|
||||
async ingest(sessionId: string, toolPairs: ToolPairPayload[]): Promise<void> {
|
||||
return this.ingestWithContext(sessionId, toolPairs, undefined, undefined);
|
||||
}
|
||||
|
||||
/**
|
||||
* Fire-and-forget: send tool pairs + optional context to ingest endpoint.
|
||||
* When prompt/recentMessages are provided, server triggers L1 with context (skip L1.5).
|
||||
*/
|
||||
async ingestWithContext(
|
||||
sessionId: string,
|
||||
toolPairs: ToolPairPayload[],
|
||||
prompt?: string,
|
||||
recentMessages?: RecentMessage[],
|
||||
): Promise<void> {
|
||||
const url = `${this.config.serverUrl}/v2/offload/ingest`;
|
||||
const payload: Record<string, unknown> = {
|
||||
session_id: sessionId,
|
||||
tool_pairs: toolPairs.map((tp) => ({
|
||||
tool_name: tp.toolName,
|
||||
tool_call_id: tp.toolCallId,
|
||||
params: tp.params,
|
||||
result: tp.result,
|
||||
error: tp.error,
|
||||
timestamp: tp.timestamp,
|
||||
duration_ms: tp.durationMs,
|
||||
})),
|
||||
};
|
||||
if (prompt) payload.prompt = prompt;
|
||||
if (recentMessages && recentMessages.length > 0) payload.recent_messages = recentMessages;
|
||||
const body = JSON.stringify(payload);
|
||||
|
||||
try {
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), this.config.ingestTimeoutMs);
|
||||
|
||||
await fetch(url, {
|
||||
method: "POST",
|
||||
headers: this.buildHeaders(),
|
||||
body,
|
||||
signal: controller.signal,
|
||||
});
|
||||
|
||||
clearTimeout(timer);
|
||||
} catch (err) {
|
||||
this.logger.warn(`[offload-client] ingest failed: ${err}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Fire-and-forget: trigger L1.5 task judgment via ingest endpoint.
|
||||
* Sends prompt + recentMessages (empty toolPairs) to activate the L1.5 path on the server.
|
||||
*/
|
||||
async ingestL15(sessionId: string, prompt: string, recentMessages?: RecentMessage[]): Promise<void> {
|
||||
const url = `${this.config.serverUrl}/v2/offload/ingest`;
|
||||
const payload: Record<string, unknown> = {
|
||||
session_id: sessionId,
|
||||
tool_pairs: [],
|
||||
prompt,
|
||||
};
|
||||
if (recentMessages && recentMessages.length > 0) payload.recent_messages = recentMessages;
|
||||
const body = JSON.stringify(payload);
|
||||
|
||||
try {
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), this.config.ingestTimeoutMs);
|
||||
|
||||
const response = await fetch(url, {
|
||||
method: "POST",
|
||||
headers: this.buildHeaders(),
|
||||
body,
|
||||
signal: controller.signal,
|
||||
});
|
||||
|
||||
clearTimeout(timer);
|
||||
|
||||
if (!response.ok) {
|
||||
this.logger.warn(`[offload-client] ingestL15 returned ${response.status}`);
|
||||
}
|
||||
} catch (err) {
|
||||
this.logger.warn(`[offload-client] ingestL15 failed: ${err}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Synchronous compaction call. Returns compressed messages + report.
|
||||
* Returns null on timeout/failure (caller should keep original messages).
|
||||
*/
|
||||
async compaction(req: {
|
||||
sessionId: string;
|
||||
messages: any[];
|
||||
ratio: number;
|
||||
contextWindow: number;
|
||||
totalTokens: number;
|
||||
messageTokens?: number[];
|
||||
}): Promise<CompactionResult | null> {
|
||||
const url = `${this.config.serverUrl}/v2/offload/compact`;
|
||||
const body = JSON.stringify({
|
||||
session_id: req.sessionId,
|
||||
messages: req.messages,
|
||||
ratio: req.ratio,
|
||||
context_window: req.contextWindow,
|
||||
total_tokens: req.totalTokens,
|
||||
message_tokens: req.messageTokens,
|
||||
});
|
||||
|
||||
try {
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), this.config.compactionTimeoutMs);
|
||||
|
||||
const response = await fetch(url, {
|
||||
method: "POST",
|
||||
headers: this.buildHeaders(),
|
||||
body,
|
||||
signal: controller.signal,
|
||||
});
|
||||
|
||||
clearTimeout(timer);
|
||||
|
||||
if (!response.ok) {
|
||||
this.logger.warn(`[offload-client] compaction returned ${response.status}`);
|
||||
return null;
|
||||
}
|
||||
|
||||
const json = (await response.json()) as any;
|
||||
if (json.code !== 0 || !json.data) {
|
||||
this.logger.warn(`[offload-client] compaction error: ${json.message ?? "unknown"}`);
|
||||
return null;
|
||||
}
|
||||
|
||||
return { messages: json.data.messages, report: json.data.report };
|
||||
} catch (err) {
|
||||
this.logger.warn(`[offload-client] compaction failed: ${err}`);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private buildHeaders(): Record<string, string> {
|
||||
return {
|
||||
"Content-Type": "application/json",
|
||||
Authorization: `Bearer ${this.config.apiKey}`,
|
||||
"X-TDAI-Service-Id": this.config.serviceId,
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,227 @@
|
||||
/**
|
||||
* offload-client — Token estimation using tiktoken (precise).
|
||||
* Aligned with server-side preciseMessageTokens: only counts LLM-visible content.
|
||||
*
|
||||
* Strategy:
|
||||
* - Primary: tiktoken BPE encoding (o200k_base) on role + content
|
||||
* - Fallback: CJK-aware heuristic if tiktoken fails
|
||||
*
|
||||
* `estimateAllTokens` still supports optional calibration from framework-reported
|
||||
* totalTokens, but with tiktoken the drift should be minimal.
|
||||
*/
|
||||
import { getEncoding, type Tiktoken } from "js-tiktoken";
|
||||
|
||||
// ─── Tiktoken Encoder (lazy singleton) ──────────────────────────────────────
|
||||
|
||||
let _encoder: Tiktoken | null = null;
|
||||
function getEncoder(): Tiktoken {
|
||||
if (!_encoder) _encoder = getEncoding("o200k_base");
|
||||
return _encoder;
|
||||
}
|
||||
|
||||
// ─── Calibration constants ──────────────────────────────────────────────────
|
||||
|
||||
/** If heuristic drifts > 15% from known total, apply linear scaling. */
|
||||
const CALIBRATION_THRESHOLD = 0.15;
|
||||
/** Clamp calibration factor to prevent extreme scaling from noisy estimates. */
|
||||
const CALIBRATION_FACTOR_MIN = 0.5;
|
||||
const CALIBRATION_FACTOR_MAX = 3.0;
|
||||
|
||||
// ─── LLM-visible text extraction (must match server-side extractLlmVisibleText) ──
|
||||
|
||||
/**
|
||||
* Extract the LLM-visible portion of a message (role + content only).
|
||||
* Matches server-side preciseMessageTokens logic exactly.
|
||||
*/
|
||||
function extractLlmVisibleText(msg: any): string {
|
||||
const role: string = msg?.role ?? msg?.message?.role ?? "";
|
||||
const rawContent = msg?.content ?? msg?.message?.content ?? "";
|
||||
|
||||
let contentStr: string;
|
||||
if (typeof rawContent === "string") {
|
||||
contentStr = rawContent;
|
||||
} else if (Array.isArray(rawContent)) {
|
||||
const parts: string[] = [];
|
||||
for (const block of rawContent) {
|
||||
if (typeof block === "string") {
|
||||
parts.push(block);
|
||||
} else if (block?.type === "text" && typeof block.text === "string") {
|
||||
parts.push(block.text);
|
||||
} else if (block?.type === "tool_use" || block?.type === "toolCall") {
|
||||
parts.push(block.name ?? block.toolName ?? "");
|
||||
if (block.arguments) {
|
||||
parts.push(typeof block.arguments === "string" ? block.arguments : JSON.stringify(block.arguments));
|
||||
}
|
||||
if (block.input) {
|
||||
parts.push(typeof block.input === "string" ? block.input : JSON.stringify(block.input));
|
||||
}
|
||||
} else if (block?.type === "tool_result") {
|
||||
if (typeof block.content === "string") parts.push(block.content);
|
||||
else if (block.content) parts.push(JSON.stringify(block.content));
|
||||
} else {
|
||||
parts.push(JSON.stringify(block));
|
||||
}
|
||||
}
|
||||
contentStr = parts.join("\n");
|
||||
} else {
|
||||
contentStr = JSON.stringify(rawContent);
|
||||
}
|
||||
|
||||
return `${role}\n${contentStr}`;
|
||||
}
|
||||
|
||||
// ─── Per-message token cache (WeakMap — auto GC when msg object is released) ──
|
||||
|
||||
const _tokenCache = new WeakMap<object, number>();
|
||||
|
||||
/** Max messages to tiktoken precisely per call. Beyond this, old messages use heuristic. */
|
||||
const PRECISE_BUDGET = 200;
|
||||
|
||||
// ─── Core estimation ────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Estimate tokens for a text string using tiktoken.
|
||||
* Falls back to CJK-aware heuristic on error.
|
||||
*/
|
||||
export function estimateTokens(text: string): number {
|
||||
if (!text) return 0;
|
||||
try {
|
||||
return getEncoder().encode(text).length;
|
||||
} catch {
|
||||
return heuristicTokens(text);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Estimate tokens for a single message using tiktoken (with WeakMap cache).
|
||||
* Counts only LLM-visible content (role + content), aligned with server.
|
||||
*/
|
||||
export function estimateMessageTokens(msg: any): number {
|
||||
if (msg == null) return 0;
|
||||
// Cache hit
|
||||
if (typeof msg === "object" && _tokenCache.has(msg)) {
|
||||
return _tokenCache.get(msg)!;
|
||||
}
|
||||
const tokens = _computeMessageTokens(msg);
|
||||
// Cache store (only for objects)
|
||||
if (typeof msg === "object" && msg !== null) {
|
||||
_tokenCache.set(msg, tokens);
|
||||
}
|
||||
return tokens;
|
||||
}
|
||||
|
||||
function _computeMessageTokens(msg: any): number {
|
||||
try {
|
||||
const text = extractLlmVisibleText(msg);
|
||||
return getEncoder().encode(text).length + 4; // +4 for message framing overhead
|
||||
} catch {
|
||||
return heuristicTokens(extractLlmVisibleText(msg));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Fast heuristic estimate for a message (no tiktoken, ~0.01ms per msg).
|
||||
* Used for old messages when total count exceeds PRECISE_BUDGET.
|
||||
*/
|
||||
function heuristicMessageTokens(msg: any): number {
|
||||
if (msg == null) return 0;
|
||||
return heuristicTokens(extractLlmVisibleText(msg));
|
||||
}
|
||||
|
||||
// ─── Calibrated batch estimation ────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Estimate total tokens and per-message tokens for a message array.
|
||||
*
|
||||
* Performance strategy:
|
||||
* - If messages.length <= PRECISE_BUDGET (200): tiktoken all (with cache)
|
||||
* - If messages.length > PRECISE_BUDGET: heuristic for old, tiktoken for recent N
|
||||
* Then calibrate old estimates using recent tiktoken as reference.
|
||||
*
|
||||
* @param knownTotalTokens Optional authoritative total from the framework.
|
||||
*/
|
||||
export function estimateAllTokens(
|
||||
messages: any[],
|
||||
knownTotalTokens?: number,
|
||||
): { total: number; perMessage: number[] } {
|
||||
const n = messages.length;
|
||||
if (n === 0) return { total: 0, perMessage: [] };
|
||||
|
||||
let raw: number[];
|
||||
|
||||
if (n <= PRECISE_BUDGET) {
|
||||
// Small batch: tiktoken all (cache makes repeat calls fast)
|
||||
raw = messages.map((msg) => estimateMessageTokens(msg));
|
||||
} else {
|
||||
// Large batch: tiktoken recent, heuristic old, calibrate
|
||||
const preciseStart = n - PRECISE_BUDGET;
|
||||
raw = new Array(n);
|
||||
|
||||
// Recent messages: precise (also populates cache for next call)
|
||||
let preciseSum = 0;
|
||||
let heuristicSumForRecent = 0;
|
||||
for (let i = preciseStart; i < n; i++) {
|
||||
raw[i] = estimateMessageTokens(messages[i]);
|
||||
preciseSum += raw[i];
|
||||
heuristicSumForRecent += heuristicMessageTokens(messages[i]);
|
||||
}
|
||||
|
||||
// Calibration factor from recent messages
|
||||
const calibFactor = heuristicSumForRecent > 0 ? preciseSum / heuristicSumForRecent : 1;
|
||||
|
||||
// Old messages: heuristic × calibration factor (or cache hit if available)
|
||||
for (let i = 0; i < preciseStart; i++) {
|
||||
if (typeof messages[i] === "object" && _tokenCache.has(messages[i])) {
|
||||
raw[i] = _tokenCache.get(messages[i])!;
|
||||
} else {
|
||||
raw[i] = Math.max(1, Math.round(heuristicMessageTokens(messages[i]) * calibFactor));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let rawTotal = raw.reduce((s, v) => s + v, 0);
|
||||
|
||||
// External calibration (from framework totalTokens)
|
||||
if (!knownTotalTokens || knownTotalTokens <= 0 || rawTotal <= 0) {
|
||||
return { total: rawTotal, perMessage: raw };
|
||||
}
|
||||
|
||||
const drift = Math.abs(knownTotalTokens - rawTotal) / knownTotalTokens;
|
||||
if (drift <= CALIBRATION_THRESHOLD) {
|
||||
return { total: rawTotal, perMessage: raw };
|
||||
}
|
||||
|
||||
// Apply linear calibration factor, clamped
|
||||
const factor = Math.max(
|
||||
CALIBRATION_FACTOR_MIN,
|
||||
Math.min(CALIBRATION_FACTOR_MAX, knownTotalTokens / rawTotal),
|
||||
);
|
||||
const calibrated = raw.map((v) => Math.max(1, Math.round(v * factor)));
|
||||
const calibratedTotal = calibrated.reduce((s, v) => s + v, 0);
|
||||
|
||||
return { total: calibratedTotal, perMessage: calibrated };
|
||||
}
|
||||
|
||||
// ─── Heuristic fallback ─────────────────────────────────────────────────────
|
||||
|
||||
function countCjkChars(text: string): number {
|
||||
let n = 0;
|
||||
for (const ch of text) {
|
||||
const c = ch.codePointAt(0)!;
|
||||
if (
|
||||
(c >= 0x4e00 && c <= 0x9fff) ||
|
||||
(c >= 0x3400 && c <= 0x4dbf) ||
|
||||
(c >= 0xf900 && c <= 0xfaff)
|
||||
) {
|
||||
n++;
|
||||
}
|
||||
}
|
||||
return n;
|
||||
}
|
||||
|
||||
function heuristicTokens(text: string): number {
|
||||
if (!text) return 0;
|
||||
const cjk = countCjkChars(text);
|
||||
const rest = Math.max(0, text.length - cjk);
|
||||
return Math.max(1, Math.ceil(cjk / 1.7 + rest / 4));
|
||||
}
|
||||
@@ -0,0 +1,80 @@
|
||||
/**
|
||||
* offload-client — Type definitions.
|
||||
*/
|
||||
|
||||
// ─── Plugin Configuration ────────────────────────────────────────────────────
|
||||
|
||||
export interface OffloadClientConfig {
|
||||
enabled: boolean;
|
||||
/** Offload server base URL (e.g. "http://localhost:9100"). */
|
||||
serverUrl: string;
|
||||
/** Bearer token for Authorization header. */
|
||||
apiKey: string;
|
||||
/** X-TDAI-Service-Id header value. */
|
||||
serviceId: string;
|
||||
/** Agent name for sessionId construction. Default "default". */
|
||||
agentName?: string;
|
||||
/** Client-side threshold: skip compaction request when ratio < this value. Default 0.5. */
|
||||
compactionRatio: number;
|
||||
/** Ingest request timeout in ms. Default 5000. */
|
||||
ingestTimeoutMs: number;
|
||||
/** Compaction request timeout in ms. Default 30000. */
|
||||
compactionTimeoutMs: number;
|
||||
}
|
||||
|
||||
export function defaultOffloadClientConfig(): OffloadClientConfig {
|
||||
return {
|
||||
enabled: false,
|
||||
serverUrl: "http://localhost:9100",
|
||||
apiKey: "",
|
||||
serviceId: "",
|
||||
compactionRatio: 0.5,
|
||||
ingestTimeoutMs: 5000,
|
||||
compactionTimeoutMs: 30000,
|
||||
};
|
||||
}
|
||||
|
||||
// ─── Ingest Payload ──────────────────────────────────────────────────────────
|
||||
|
||||
export interface ToolPairPayload {
|
||||
toolName: string;
|
||||
toolCallId: string;
|
||||
params: unknown;
|
||||
result: unknown;
|
||||
error?: string;
|
||||
timestamp: string;
|
||||
durationMs?: number;
|
||||
}
|
||||
|
||||
export interface RecentMessage {
|
||||
role: "user" | "assistant";
|
||||
content: string;
|
||||
}
|
||||
|
||||
// ─── Compaction Response ─────────────────────────────────────────────────────
|
||||
|
||||
export interface CompactionReport {
|
||||
resolvedLevel: string;
|
||||
originalCount: number;
|
||||
compactedCount: number;
|
||||
fastPathReplaced: number;
|
||||
fastPathDeleted: number;
|
||||
mildReplacements: number;
|
||||
aggressiveDeleted: number;
|
||||
emergencyDeleted: number;
|
||||
mmdInjected: number;
|
||||
}
|
||||
|
||||
export interface CompactionResult {
|
||||
messages: any[];
|
||||
report: CompactionReport;
|
||||
}
|
||||
|
||||
// ─── Logger ──────────────────────────────────────────────────────────────────
|
||||
|
||||
export interface Logger {
|
||||
info: (...args: unknown[]) => void;
|
||||
warn: (...args: unknown[]) => void;
|
||||
error: (...args: unknown[]) => void;
|
||||
debug?: (...args: unknown[]) => void;
|
||||
}
|
||||
@@ -27,6 +27,8 @@ export interface CallLlmOpts {
|
||||
timeoutMs?: number;
|
||||
/** Label for logging (e.g. "L1", "L1.5", "L2") */
|
||||
label?: string;
|
||||
/** Instance ID for telemetry metadata */
|
||||
instanceId?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -64,6 +66,7 @@ export async function callLlm(
|
||||
experimental_telemetry: {
|
||||
isEnabled: true,
|
||||
functionId: opts.label ?? "offload-llm",
|
||||
metadata: { instanceId: opts.instanceId ?? "unknown" },
|
||||
},
|
||||
});
|
||||
|
||||
|
||||
@@ -3,6 +3,8 @@
|
||||
* Ported from context-offload-plugin with updated runtime defaults.
|
||||
*/
|
||||
|
||||
import type { Logger } from "../core/types.js";
|
||||
|
||||
// ============================
|
||||
// Data types
|
||||
// ============================
|
||||
@@ -212,12 +214,7 @@ export interface PluginConfig {
|
||||
// ============================
|
||||
|
||||
/** Logger interface used by offload plugin components */
|
||||
export interface PluginLogger {
|
||||
info: (msg: string) => void;
|
||||
warn: (msg: string) => void;
|
||||
error: (msg: string) => void;
|
||||
debug?: (msg: string) => void;
|
||||
}
|
||||
export type PluginLogger = Logger;
|
||||
|
||||
// ============================
|
||||
// Plugin defaults
|
||||
|
||||
@@ -0,0 +1,328 @@
|
||||
/**
|
||||
* L3 Compaction Handler — orchestrates fast-path, MMD injection, and compression.
|
||||
* Synchronous API: client awaits the response.
|
||||
*
|
||||
* Uses compact-state.json (independent from state.json) to avoid lock contention
|
||||
* with L1/L1.5/L2 executors that write state.json concurrently.
|
||||
*/
|
||||
import type http from "node:http";
|
||||
import type { StorageAdapter } from "../../core/storage/adapter.js";
|
||||
import type { OffloadEntry, OffloadState, OffloadExecutorConfig, CompactState } from "../types.js";
|
||||
import { defaultOffloadState, defaultCompactState } from "../types.js";
|
||||
import { parseJsonl } from "../parsers/json-utils.js";
|
||||
import { CompactionRequestSchemaV2 } from "../schemas.js";
|
||||
import { buildOffloadBasePath } from "../session-utils.js";
|
||||
import { applyFastPath } from "./fast-path.js";
|
||||
import { injectActiveMmd, injectHistoryMmds } from "./mmd-injector.js";
|
||||
import { resolveLevel, mildCompress, aggressiveCompress, emergencyCompress } from "./compressor.js";
|
||||
import { estimateMessageTokens, extractToolResultId } from "./helpers.js";
|
||||
import type { Message } from "./helpers.js";
|
||||
import { traceServerCompaction } from "../opik-tracer.js";
|
||||
|
||||
export interface CompactionDeps {
|
||||
storage: StorageAdapter;
|
||||
config: OffloadExecutorConfig;
|
||||
logger: { info: (...args: unknown[]) => void; warn: (...args: unknown[]) => void; error: (...args: unknown[]) => void };
|
||||
}
|
||||
|
||||
export interface CompactionReport {
|
||||
resolvedLevel: string;
|
||||
originalCount: number;
|
||||
compactedCount: number;
|
||||
fastPathReplaced: number;
|
||||
fastPathDeleted: number;
|
||||
mildReplacements: number;
|
||||
aggressiveDeleted: number;
|
||||
emergencyDeleted: number;
|
||||
mmdInjected: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle POST /v2/offload/compact.
|
||||
*/
|
||||
export async function handleCompaction(
|
||||
req: http.IncomingMessage,
|
||||
res: http.ServerResponse,
|
||||
auth: { serviceId: string },
|
||||
deps: CompactionDeps,
|
||||
requestId: string,
|
||||
parseJsonBody: <T>(req: http.IncomingMessage) => Promise<T>,
|
||||
sendJson: (res: http.ServerResponse, status: number, body: unknown) => void,
|
||||
successEnvelope: <T>(data: T, requestId: string) => unknown,
|
||||
errorEnvelope: (code: number, message: string, requestId: string) => unknown,
|
||||
): Promise<void> {
|
||||
const body = await parseJsonBody(req);
|
||||
const parsed = CompactionRequestSchemaV2.safeParse(body);
|
||||
if (!parsed.success) {
|
||||
sendJson(res, 400, errorEnvelope(400, parsed.error.message, requestId));
|
||||
return;
|
||||
}
|
||||
|
||||
const { session_id: sessionId, messages, ratio, context_window: contextWindow, message_tokens: messageTokens } = parsed.data;
|
||||
let { total_tokens: totalTokens } = parsed.data;
|
||||
const { storage, config } = deps;
|
||||
const basePath = buildOffloadBasePath(sessionId);
|
||||
const originalCount = messages.length;
|
||||
const compactionStartMs = Date.now();
|
||||
|
||||
const { preciseMessageTokens } = await import("./compressor.js");
|
||||
const originalTotalTokens = totalTokens;
|
||||
|
||||
// Compute fixed overhead: system prompt + tool schemas + message framing.
|
||||
// These are included in clientTotalTokens but NOT in messages.
|
||||
// overhead = clientTotal - sum(preciseMessageTokens for each message)
|
||||
const messagesTokenSum = (messages as any[]).reduce(
|
||||
(s: number, msg: any) => s + preciseMessageTokens(msg), 0,
|
||||
);
|
||||
const fixedOverhead = Math.max(0, totalTokens - messagesTokenSum);
|
||||
|
||||
deps.logger.info(
|
||||
`[offload-server] compaction: clientTotal=${totalTokens}, msgsTokens=${messagesTokenSum}, ` +
|
||||
`overhead=${fixedOverhead}, msgs=${originalCount}, ratio=${ratio.toFixed(2)}`,
|
||||
);
|
||||
|
||||
// Read offload state (L1/L1.5/L2 managed, read-only here) and compact state (L3 owned)
|
||||
const state = await readOffloadState(storage, basePath);
|
||||
const compactState = await readCompactState(storage, basePath);
|
||||
const entriesRaw = await storage.readFile(`${basePath}/entries.jsonl`);
|
||||
const entries = entriesRaw
|
||||
? parseJsonl<OffloadEntry>(entriesRaw, (line, err) => {
|
||||
deps.logger.warn(`[offload-server] compaction: bad JSONL line: ${line}`, err);
|
||||
})
|
||||
: [];
|
||||
|
||||
// Merge node-mapping.jsonl into entries (L2 writes node_id to a separate file)
|
||||
const nodeMappingRaw = await storage.readFile(`${basePath}/node-mapping.jsonl`);
|
||||
if (nodeMappingRaw) {
|
||||
const mappings = parseJsonl<{ tool_call_id: string; node_id: string }>(nodeMappingRaw);
|
||||
const nodeMap = new Map(mappings.map((m) => [m.tool_call_id, m.node_id]));
|
||||
for (const entry of entries) {
|
||||
if (!entry.node_id && nodeMap.has(entry.tool_call_id)) {
|
||||
entry.node_id = nodeMap.get(entry.tool_call_id)!;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Token array: use tiktoken values directly (no calibration against clientTotal).
|
||||
// clientTotal includes fixedOverhead which is not in messages, so calibration
|
||||
// would inflate per-message tokens incorrectly.
|
||||
const tokenArray = buildTokenArray(messages as Message[], messagesTokenSum, messageTokens);
|
||||
|
||||
const report: CompactionReport = {
|
||||
resolvedLevel: "fastpath",
|
||||
originalCount,
|
||||
compactedCount: 0,
|
||||
fastPathReplaced: 0,
|
||||
fastPathDeleted: 0,
|
||||
mildReplacements: 0,
|
||||
aggressiveDeleted: 0,
|
||||
emergencyDeleted: 0,
|
||||
mmdInjected: 0,
|
||||
};
|
||||
|
||||
// Step 1: Fast-path re-apply (uses compactState for confirmed/deleted IDs)
|
||||
const fp = applyFastPath(messages, entries, compactState);
|
||||
report.fastPathReplaced = fp.replacedCount;
|
||||
report.fastPathDeleted = fp.deletedCount;
|
||||
|
||||
// Recalculate totalTokens after fast-path:
|
||||
// totalTokens = tiktoken(remaining messages) + fixedOverhead
|
||||
if (fp.deletedCount > 0) {
|
||||
const postFpMsgsTokens = (messages as any[]).reduce(
|
||||
(s: number, msg: any) => s + preciseMessageTokens(msg), 0,
|
||||
);
|
||||
totalTokens = postFpMsgsTokens + fixedOverhead;
|
||||
// Rebuild tokenArray for remaining messages
|
||||
tokenArray.length = 0;
|
||||
tokenArray.push(...buildTokenArray(messages as Message[], postFpMsgsTokens, undefined));
|
||||
}
|
||||
const effectiveRatio = contextWindow > 0 ? totalTokens / contextWindow : ratio;
|
||||
|
||||
// Step 2: Resolve compression level using post-fast-path ratio
|
||||
const level = resolveLevel(effectiveRatio, {
|
||||
mildRatio: config.mildOffloadRatio,
|
||||
aggressiveRatio: config.aggressiveCompressRatio,
|
||||
emergencyRatio: config.emergencyCompressRatio,
|
||||
});
|
||||
report.resolvedLevel = level;
|
||||
|
||||
deps.logger.info(
|
||||
`[offload-server] compaction: level=${level}, ratio=${effectiveRatio.toFixed(2)} (pre-fp=${ratio.toFixed(2)}), msgs=${messages.length}, entries=${entries.length}`,
|
||||
);
|
||||
|
||||
// Step 3: Mild compression
|
||||
if (level === "mild" || level === "aggressive" || level === "emergency") {
|
||||
const mild = mildCompress(messages, entries);
|
||||
report.mildReplacements = mild.replacedCount;
|
||||
compactState.confirmedOffloadIds.push(...mild.confirmedIds);
|
||||
|
||||
// Sync tokenArray after mild replacements (content changed)
|
||||
// Incremental update: only recalculate tokens for messages that were actually replaced
|
||||
if (mild.replacedCount > 0) {
|
||||
const confirmedSet = new Set(mild.confirmedIds);
|
||||
let tokenDelta = 0;
|
||||
for (let i = 0; i < messages.length; i++) {
|
||||
const msg = messages[i] as any;
|
||||
// Use extractToolResultId which handles all formats (OpenAI, Anthropic content blocks)
|
||||
const tid = extractToolResultId(msg);
|
||||
if (!tid || !confirmedSet.has(tid)) continue;
|
||||
const newTokens = preciseMessageTokens(msg);
|
||||
tokenDelta += newTokens - tokenArray[i];
|
||||
tokenArray[i] = newTokens;
|
||||
}
|
||||
totalTokens += tokenDelta;
|
||||
}
|
||||
}
|
||||
|
||||
// Step 4: Aggressive compression
|
||||
// Target: just below the aggressive trigger threshold (leave ~5% headroom)
|
||||
let aggRemainingTokens = totalTokens; // track for emergency
|
||||
if (level === "aggressive" || level === "emergency") {
|
||||
const aggTargetTokens = Math.floor(contextWindow * (config.aggressiveCompressRatio - 0.05));
|
||||
// Skip aggressive if mild already brought tokens below target
|
||||
if (totalTokens <= aggTargetTokens) {
|
||||
deps.logger.info(
|
||||
`[offload-server] aggressive skipped: mild already reduced tokens to ${totalTokens} (target=${aggTargetTokens})`,
|
||||
);
|
||||
} else {
|
||||
const agg = aggressiveCompress(messages, aggTargetTokens, tokenArray, totalTokens);
|
||||
report.aggressiveDeleted = agg.deletedCount;
|
||||
aggRemainingTokens = agg.remainingTokens;
|
||||
compactState.deletedOffloadIds.push(...agg.deletedIds);
|
||||
|
||||
// Inject history MMDs for deleted entries
|
||||
if (agg.deletedIds.length > 0) {
|
||||
const mmdBudget = Math.floor(contextWindow * 0.1 / 4); // 10% of context, in estimated tokens
|
||||
const hist = await injectHistoryMmds(
|
||||
messages, agg.deletedIds, entries, state, storage, basePath, mmdBudget,
|
||||
);
|
||||
report.mmdInjected += hist.injectedCount;
|
||||
// Sync tokenArray for any injected MMD messages (calibrate against messages-only tokens)
|
||||
if (hist.injectedCount > 0) {
|
||||
const postAggMsgsTokens = (messages as any[]).reduce(
|
||||
(s: number, msg: any) => s + preciseMessageTokens(msg), 0,
|
||||
);
|
||||
tokenArray.length = 0;
|
||||
tokenArray.push(...buildTokenArray(messages as Message[], postAggMsgsTokens, undefined));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Step 6: Emergency compression
|
||||
// Target: just below the aggressive threshold (so next turn won't immediately re-trigger)
|
||||
if (level === "emergency") {
|
||||
const emTargetTokens = Math.floor(contextWindow * (config.aggressiveCompressRatio - 0.10));
|
||||
const em = emergencyCompress(messages, emTargetTokens, tokenArray, aggRemainingTokens);
|
||||
report.emergencyDeleted = em.deletedCount;
|
||||
aggRemainingTokens = em.remainingTokens;
|
||||
compactState.deletedOffloadIds.push(...em.deletedIds);
|
||||
}
|
||||
|
||||
// Step 7: Inject active MMD (after all compression, so position is correct)
|
||||
const mmdResult = await injectActiveMmd(messages, state, storage, basePath);
|
||||
report.mmdInjected += mmdResult.injectedCount;
|
||||
|
||||
// Step 8: Write compact-state.json (independent file, no lock needed)
|
||||
report.compactedCount = messages.length;
|
||||
compactState.lastCompactedAt = new Date().toISOString();
|
||||
await writeCompactState(storage, basePath, compactState);
|
||||
|
||||
// Compute remaining tokens (tracked from aggressive/emergency, no full re-scan)
|
||||
const remainingTokens = level === "fastpath" || level === "mild"
|
||||
? totalTokens // no deletion happened
|
||||
: aggRemainingTokens; // tracked through aggressive → emergency chain
|
||||
const remainingRatio = contextWindow > 0 ? (remainingTokens / contextWindow).toFixed(2) : "N/A";
|
||||
|
||||
// Opik trace: compaction decision
|
||||
traceServerCompaction({
|
||||
sessionId,
|
||||
level,
|
||||
ratio,
|
||||
contextWindow,
|
||||
totalTokensBefore: originalTotalTokens,
|
||||
totalTokensAfter: remainingTokens,
|
||||
originalMsgCount: originalCount,
|
||||
compactedMsgCount: messages.length,
|
||||
report: report as unknown as Record<string, unknown>,
|
||||
messages: messages as unknown[],
|
||||
durationMs: Date.now() - compactionStartMs,
|
||||
logger: deps.logger,
|
||||
});
|
||||
|
||||
// Step 8: Return
|
||||
sendJson(res, 200, successEnvelope({ messages, report }, requestId));
|
||||
|
||||
deps.logger.info(
|
||||
`[offload-server] compaction done: ${originalCount}→${messages.length} msgs, level=${level}, ` +
|
||||
`tokens=${originalTotalTokens}→${remainingTokens} (${remainingRatio}), ` +
|
||||
`fp=${fp.replacedCount}r/${fp.deletedCount}d, mild=${report.mildReplacements}, ` +
|
||||
`agg=${report.aggressiveDeleted}, em=${report.emergencyDeleted}, mmd=${report.mmdInjected}`,
|
||||
);
|
||||
}
|
||||
|
||||
// ─── Token Array Builder ─────────────────────────────────────────────────────
|
||||
|
||||
/** Calibration threshold: if heuristic estimate drifts >15% from API totalTokens, apply linear scaling. */
|
||||
const CALIBRATION_THRESHOLD = 0.15;
|
||||
/** Clamp calibration factor to prevent extreme scaling from noisy estimates. */
|
||||
const CALIBRATION_FACTOR_MIN = 0.5;
|
||||
const CALIBRATION_FACTOR_MAX = 3.0;
|
||||
|
||||
/**
|
||||
* Build a pre-computed token array for all messages, with optional linear calibration.
|
||||
* - If messageTokens[i] is available, use it directly (precise value).
|
||||
* - Otherwise, use estimateMessageTokens (CJK-aware heuristic).
|
||||
* - If totalTokens is provided and drift > 15%, apply a calibration factor to estimated items.
|
||||
*/
|
||||
export function buildTokenArray(
|
||||
messages: Message[],
|
||||
totalTokens: number,
|
||||
messageTokens?: number[],
|
||||
): number[] {
|
||||
const raw = messages.map((msg, i) =>
|
||||
messageTokens && i < messageTokens.length
|
||||
? messageTokens[i]
|
||||
: estimateMessageTokens(msg),
|
||||
);
|
||||
|
||||
const rawTotal = raw.reduce((s, v) => s + v, 0);
|
||||
if (rawTotal <= 0 || totalTokens <= 0) return raw;
|
||||
|
||||
const drift = Math.abs(totalTokens - rawTotal) / totalTokens;
|
||||
if (drift <= CALIBRATION_THRESHOLD) return raw;
|
||||
|
||||
// Linear calibration: only scale estimated items, keep precise items unchanged
|
||||
const factor = Math.max(CALIBRATION_FACTOR_MIN, Math.min(CALIBRATION_FACTOR_MAX, totalTokens / rawTotal));
|
||||
return raw.map((v, i) =>
|
||||
messageTokens && i < messageTokens.length
|
||||
? v
|
||||
: Math.max(1, Math.round(v * factor)),
|
||||
);
|
||||
}
|
||||
|
||||
// ─── State Helpers ───────────────────────────────────────────────────────────
|
||||
|
||||
async function readOffloadState(storage: StorageAdapter, basePath: string): Promise<OffloadState> {
|
||||
const raw = await storage.readFile(`${basePath}/state.json`);
|
||||
if (!raw) return defaultOffloadState();
|
||||
try {
|
||||
return { ...defaultOffloadState(), ...JSON.parse(raw) };
|
||||
} catch {
|
||||
return defaultOffloadState();
|
||||
}
|
||||
}
|
||||
|
||||
async function readCompactState(storage: StorageAdapter, basePath: string): Promise<CompactState> {
|
||||
const raw = await storage.readFile(`${basePath}/compact-state.json`);
|
||||
if (!raw) return defaultCompactState();
|
||||
try {
|
||||
return { ...defaultCompactState(), ...JSON.parse(raw) };
|
||||
} catch {
|
||||
return defaultCompactState();
|
||||
}
|
||||
}
|
||||
|
||||
async function writeCompactState(storage: StorageAdapter, basePath: string, state: CompactState): Promise<void> {
|
||||
await storage.writeFile(`${basePath}/compact-state.json`, JSON.stringify(state));
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,189 @@
|
||||
/**
|
||||
* L3 Fast-path — re-apply confirmed/deleted offload state to messages.
|
||||
* Runs on every compaction call as the first step.
|
||||
*/
|
||||
import type { OffloadEntry, CompactState } from "../types.js";
|
||||
import type { Message } from "./helpers.js";
|
||||
import {
|
||||
extractToolResultId,
|
||||
isToolResultMessage,
|
||||
isOnlyToolUseAssistant,
|
||||
isAssistantWithToolUse,
|
||||
extractAllToolUseIds,
|
||||
replaceWithSummary,
|
||||
buildOffloadMap,
|
||||
} from "./helpers.js";
|
||||
|
||||
export interface FastPathResult {
|
||||
replacedCount: number;
|
||||
deletedCount: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Apply fast-path: replace confirmed tool results with summary,
|
||||
* delete messages whose tool_call_id is in deletedOffloadIds,
|
||||
* strip orphaned toolCall blocks from mixed assistant messages.
|
||||
* Mutates messages in place.
|
||||
*/
|
||||
export function applyFastPath(
|
||||
messages: Message[],
|
||||
entries: OffloadEntry[],
|
||||
compactState: CompactState,
|
||||
): FastPathResult {
|
||||
const offloadMap = buildOffloadMap(entries);
|
||||
const confirmedSet = new Set(compactState.confirmedOffloadIds);
|
||||
const deletedSet = new Set(compactState.deletedOffloadIds);
|
||||
|
||||
if (confirmedSet.size === 0 && deletedSet.size === 0) {
|
||||
return { replacedCount: 0, deletedCount: 0 };
|
||||
}
|
||||
|
||||
const indicesToDelete: number[] = [];
|
||||
let replacedCount = 0;
|
||||
|
||||
for (let i = 0; i < messages.length; i++) {
|
||||
const msg = messages[i];
|
||||
const tid = extractToolResultId(msg);
|
||||
|
||||
// 1. Deleted tool_result — handle Anthropic multi-block user messages carefully
|
||||
if (tid && deletedSet.has(tid)) {
|
||||
// Anthropic format: user message may have multiple tool_result blocks
|
||||
if (msg.role === "user" && Array.isArray(msg.content)) {
|
||||
const allToolResultIds = msg.content
|
||||
.filter((b: any) => b?.type === "tool_result" && b.tool_use_id)
|
||||
.map((b: any) => b.tool_use_id as string);
|
||||
|
||||
if (allToolResultIds.length > 1) {
|
||||
// Multi-block: only remove the blocks that are in deletedSet, keep the rest
|
||||
const allDeleted = allToolResultIds.every((id: string) => deletedSet.has(id));
|
||||
if (allDeleted) {
|
||||
indicesToDelete.push(i);
|
||||
} else {
|
||||
// Partial: strip only deleted tool_result blocks
|
||||
for (let j = msg.content.length - 1; j >= 0; j--) {
|
||||
const block = msg.content[j];
|
||||
if (block?.type === "tool_result" && block.tool_use_id && deletedSet.has(block.tool_use_id)) {
|
||||
msg.content.splice(j, 1);
|
||||
}
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
}
|
||||
// Single tool_result or non-Anthropic format: delete entire message
|
||||
indicesToDelete.push(i);
|
||||
continue;
|
||||
}
|
||||
|
||||
// 2. Confirmed tool_result → replace with summary
|
||||
if (tid && confirmedSet.has(tid) && isToolResultMessage(msg) && !msg._offloaded) {
|
||||
const entry = offloadMap.get(tid);
|
||||
if (entry) {
|
||||
replaceWithSummary(msg, entry);
|
||||
replacedCount++;
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Pure toolCall assistant — all IDs deleted → mark for deletion
|
||||
if (!tid && isOnlyToolUseAssistant(msg)) {
|
||||
const tuIds = extractAllToolUseIds(msg);
|
||||
if (tuIds.length > 0 && tuIds.every((id) => deletedSet.has(id))) {
|
||||
indicesToDelete.push(i);
|
||||
}
|
||||
}
|
||||
|
||||
// 4. Mixed assistant (text + toolCall) — strip deleted toolCall blocks
|
||||
// to prevent orphaned tool_use without matching tool_result (provider 400)
|
||||
if (!tid && isAssistantWithToolUse(msg) && !isOnlyToolUseAssistant(msg)) {
|
||||
const content = msg.type === "message" ? msg.message?.content : msg.content;
|
||||
if (Array.isArray(content)) {
|
||||
for (let j = content.length - 1; j >= 0; j--) {
|
||||
const block = content[j];
|
||||
if ((block?.type === "tool_use" || block?.type === "toolCall") && block.id) {
|
||||
if (deletedSet.has(block.id)) {
|
||||
content.splice(j, 1);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Delete in reverse order to preserve indices
|
||||
const uniqueIndices = [...new Set(indicesToDelete)].sort((a, b) => b - a);
|
||||
for (const idx of uniqueIndices) {
|
||||
messages.splice(idx, 1);
|
||||
}
|
||||
|
||||
// Post-pass: ensure no orphaned tool_use or tool_result after deletion
|
||||
// Collect all remaining tool_use IDs and tool_result IDs
|
||||
const remainingToolUseIds = new Set<string>();
|
||||
const remainingToolResultIds = new Set<string>();
|
||||
for (const msg of messages) {
|
||||
for (const id of extractAllToolUseIds(msg)) {
|
||||
remainingToolUseIds.add(id);
|
||||
}
|
||||
if (msg.role === "user" && Array.isArray(msg.content)) {
|
||||
for (const block of msg.content) {
|
||||
if (block?.type === "tool_result" && block.tool_use_id) {
|
||||
remainingToolResultIds.add(block.tool_use_id);
|
||||
}
|
||||
}
|
||||
}
|
||||
const singleTid = extractToolResultId(msg);
|
||||
if (singleTid) remainingToolResultIds.add(singleTid);
|
||||
}
|
||||
|
||||
// Remove orphaned tool_use blocks from assistant messages (no matching tool_result)
|
||||
for (const msg of messages) {
|
||||
if (!isAssistantWithToolUse(msg)) continue;
|
||||
const content = msg.type === "message" ? msg.message?.content : msg.content;
|
||||
if (!Array.isArray(content)) continue;
|
||||
for (let j = content.length - 1; j >= 0; j--) {
|
||||
const block = content[j];
|
||||
if ((block?.type === "tool_use" || block?.type === "toolCall") && block.id) {
|
||||
if (!remainingToolResultIds.has(block.id)) {
|
||||
content.splice(j, 1);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Remove orphaned tool_result messages/blocks (no matching tool_use)
|
||||
for (let i = messages.length - 1; i >= 0; i--) {
|
||||
const msg = messages[i];
|
||||
if (msg.role === "user" && Array.isArray(msg.content)) {
|
||||
for (let j = msg.content.length - 1; j >= 0; j--) {
|
||||
const block = msg.content[j];
|
||||
if (block?.type === "tool_result" && block.tool_use_id && !remainingToolUseIds.has(block.tool_use_id)) {
|
||||
msg.content.splice(j, 1);
|
||||
}
|
||||
}
|
||||
// If user message is now empty, remove it
|
||||
if (msg.content.length === 0) {
|
||||
messages.splice(i, 1);
|
||||
uniqueIndices.push(i); // count it
|
||||
}
|
||||
} else {
|
||||
const singleTid = extractToolResultId(msg);
|
||||
if (singleTid && !remainingToolUseIds.has(singleTid) && isToolResultMessage(msg)) {
|
||||
messages.splice(i, 1);
|
||||
uniqueIndices.push(i);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Remove empty assistant messages (all tool_use blocks were stripped)
|
||||
for (let i = messages.length - 1; i >= 0; i--) {
|
||||
const msg = messages[i];
|
||||
const role = msg.role ?? msg.message?.role ?? msg.type;
|
||||
if (role !== "assistant") continue;
|
||||
const content = msg.type === "message" ? msg.message?.content : msg.content;
|
||||
if (Array.isArray(content) && content.length === 0) {
|
||||
messages.splice(i, 1);
|
||||
uniqueIndices.push(i);
|
||||
}
|
||||
}
|
||||
|
||||
return { replacedCount, deletedCount: uniqueIndices.length };
|
||||
}
|
||||
@@ -0,0 +1,297 @@
|
||||
/**
|
||||
* L3 Compaction — Message helper utilities.
|
||||
* Handles multiple message formats (OpenAI, Anthropic, OpenClaw).
|
||||
*
|
||||
* ═══════════════════════════════════════════════════════════════════════════
|
||||
* OpenClaw 消息格式说明(来源:@mariozechner/pi-ai 类型定义)
|
||||
* ═══════════════════════════════════════════════════════════════════════════
|
||||
*
|
||||
* OpenClaw 内部使用统一的 AgentMessage 联合类型,包含以下几种消息:
|
||||
*
|
||||
* ── 1. UserMessage ──
|
||||
* {
|
||||
* role: "user",
|
||||
* content: string | ContentBlock[], // string 或 [{type:"text",text:"..."}] 或含图片块
|
||||
* timestamp: number,
|
||||
* }
|
||||
*
|
||||
* ── 2. AssistantMessage(纯文本回复)──
|
||||
* {
|
||||
* role: "assistant",
|
||||
* content: [{ type: "text", text: "..." }],
|
||||
* model: "gpt-5.2",
|
||||
* stopReason: "stop",
|
||||
* timestamp: number,
|
||||
* api: "messages" | "chat" | "responses",
|
||||
* provider: "anthropic" | "openai" | "google",
|
||||
* usage: { input, output, totalTokens, ... },
|
||||
* }
|
||||
*
|
||||
* ── 3. AssistantMessage(含 tool_use / toolCall)──
|
||||
* {
|
||||
* role: "assistant",
|
||||
* content: [
|
||||
* { type: "text", text: "I'll read the file..." }, // 可选文本块
|
||||
* { type: "toolCall", id: "call_abc123", name: "read_file", arguments: { path: "..." } },
|
||||
* { type: "toolCall", id: "call_def456", name: "exec", arguments: { cmd: "..." } },
|
||||
* ],
|
||||
* stopReason: "toolUse",
|
||||
* ...同上
|
||||
* }
|
||||
* 注: Anthropic 原生格式使用 { type: "tool_use", id, name, input }
|
||||
* OpenClaw 内部统一为 { type: "toolCall", id, name, arguments }
|
||||
* 但消息到达后端时**两种格式都可能出现**(取决于客户端是否已转换),
|
||||
* 因此本模块所有判断都同时匹配 "tool_use" 和 "toolCall"。
|
||||
*
|
||||
* ── 4. ToolResultMessage ──
|
||||
* {
|
||||
* role: "toolResult",
|
||||
* toolCallId: "call_abc123", // 对应 AssistantMessage 中 toolCall 的 id
|
||||
* toolName: "read_file",
|
||||
* content: [{ type: "text", text: "文件内容..." }],
|
||||
* isError: false,
|
||||
* timestamp: number,
|
||||
* details?: any, // 可选的详细信息(不发给 LLM)
|
||||
* }
|
||||
*
|
||||
* ── 5. 消息配对规则 ──
|
||||
* - 每个 AssistantMessage 中的 toolCall 必须有对应的 ToolResultMessage
|
||||
* - toolCallId 是配对的唯一标识
|
||||
* - 删除 tool_result 时必须同时删除对应的 assistant toolCall(否则 provider 返回 400)
|
||||
* - 一个 assistant 消息可包含多个 toolCall(并行工具调用)
|
||||
*
|
||||
* ── 6. 转换到 LLM Provider 原生格式 ──
|
||||
* OpenAI: toolResult → { role: "tool", tool_call_id: "...", content: "..." }
|
||||
* Anthropic: toolResult → { role: "user", content: [{ type: "tool_result", tool_use_id: "...", content: "..." }] }
|
||||
* 注: 本模块处理的是 **转换后** 的消息(发送给 provider 前的格式),
|
||||
* 因此需要兼容所有上述格式。
|
||||
*
|
||||
* ── 7. 特殊标记字段(本模块使用)──
|
||||
* - _offloaded: boolean — 已被 summary 替换的 tool_result
|
||||
* - _mmdContextMessage: string — MMD 注入消息标记("active" | "history")
|
||||
* - _mmdInjection: boolean — 历史 MMD 注入消息标记
|
||||
* - _mmdVersion: string — MMD 内容哈希,用于版本去重
|
||||
* ═══════════════════════════════════════════════════════════════════════════
|
||||
*/
|
||||
import type { OffloadEntry } from "../types.js";
|
||||
|
||||
// ─── Message type aliases ────────────────────────────────────────────────────
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
export type Message = Record<string, any>;
|
||||
|
||||
// ─── Tool Result ID Extraction ───────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Extract the linked tool_use_id from a tool-result message (supports multiple formats).
|
||||
*/
|
||||
export function extractToolResultId(msg: Message): string | null {
|
||||
// OpenAI tool result: { role: "tool", tool_call_id: "..." }
|
||||
if (msg.tool_call_id) return msg.tool_call_id;
|
||||
// Anthropic tool_result: { type: "tool_result", tool_use_id: "..." }
|
||||
if (msg.tool_use_id) return msg.tool_use_id;
|
||||
// OpenClaw wrapped: { type: "message", message: { id: "...", role: "toolResult" } }
|
||||
if (msg.type === "message" && msg.message?.id) return msg.message.id;
|
||||
// Anthropic content block tool_result: { role: "user", content: [{type:"tool_result", tool_use_id}] }
|
||||
if (msg.role === "user" && Array.isArray(msg.content)) {
|
||||
for (const block of msg.content) {
|
||||
if (block?.type === "tool_result" && block.tool_use_id) return block.tool_use_id;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// ─── Message Type Checks ─────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Check if a message is a tool result (contains tool output).
|
||||
*/
|
||||
export function isToolResultMessage(msg: Message): boolean {
|
||||
if (msg.role === "tool") return true;
|
||||
if (msg.type === "tool_result") return true;
|
||||
const innerRole = msg.message?.role ?? msg.type;
|
||||
if (innerRole === "toolResult" || innerRole === "tool_result") return true;
|
||||
// Anthropic user message with tool_result content blocks
|
||||
if (msg.role === "user" && Array.isArray(msg.content)) {
|
||||
return msg.content.some((b: any) => b?.type === "tool_result");
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if a message is an assistant message containing only tool_use blocks.
|
||||
*/
|
||||
export function isOnlyToolUseAssistant(msg: Message): boolean {
|
||||
const role = msg.role ?? msg.message?.role;
|
||||
if (role !== "assistant") return false;
|
||||
const content = msg.type === "message" ? msg.message?.content : msg.content;
|
||||
if (!Array.isArray(content)) return false;
|
||||
if (content.length === 0) return false;
|
||||
return content.every((block: any) =>
|
||||
block?.type === "tool_use" || block?.type === "toolCall",
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if a message is an assistant message that contains tool_use (may also have text).
|
||||
* Supports both Anthropic format (content blocks) and OpenAI format (tool_calls field).
|
||||
*/
|
||||
export function isAssistantWithToolUse(msg: Message): boolean {
|
||||
const role = msg.role ?? msg.message?.role;
|
||||
if (role !== "assistant") return false;
|
||||
// OpenAI format: tool_calls field
|
||||
if (msg.tool_calls && Array.isArray(msg.tool_calls) && msg.tool_calls.length > 0) return true;
|
||||
// Anthropic format: content blocks
|
||||
const content = msg.type === "message" ? msg.message?.content : msg.content;
|
||||
if (!Array.isArray(content)) return false;
|
||||
return content.some((block: any) =>
|
||||
block?.type === "tool_use" || block?.type === "toolCall",
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract all tool_use IDs from an assistant message.
|
||||
* Supports both Anthropic format (content blocks) and OpenAI format (tool_calls field).
|
||||
*/
|
||||
export function extractAllToolUseIds(msg: Message): string[] {
|
||||
const ids: string[] = [];
|
||||
// OpenAI format: tool_calls field
|
||||
if (msg.tool_calls && Array.isArray(msg.tool_calls)) {
|
||||
for (const tc of msg.tool_calls) {
|
||||
if (tc.id) ids.push(tc.id);
|
||||
}
|
||||
}
|
||||
// Anthropic format: content blocks
|
||||
const content = msg.type === "message" ? msg.message?.content : msg.content;
|
||||
if (Array.isArray(content)) {
|
||||
for (const block of content) {
|
||||
if ((block?.type === "tool_use" || block?.type === "toolCall") && block.id) {
|
||||
ids.push(block.id);
|
||||
}
|
||||
}
|
||||
}
|
||||
return ids;
|
||||
}
|
||||
|
||||
// ─── Replace With Summary ────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Replace a tool result message's content with the offload summary.
|
||||
*/
|
||||
export function replaceWithSummary(msg: Message, entry: OffloadEntry): void {
|
||||
const parts = [
|
||||
`[Offloaded Tool Result | node: ${entry.node_id ?? "N/A"}]`,
|
||||
`Summary: ${entry.summary}`,
|
||||
];
|
||||
if (entry.result_ref) {
|
||||
parts.push(`原始工具结果已存档,如需查看完整内容请调用 tdai_read_cos(path="${entry.result_ref}")`);
|
||||
}
|
||||
const summaryContent = parts.join("\n");
|
||||
|
||||
if (msg.type === "message" && msg.message) {
|
||||
if (Array.isArray(msg.message.content)) {
|
||||
msg.message.content = [{ type: "text", text: summaryContent }];
|
||||
} else {
|
||||
msg.message.content = summaryContent;
|
||||
}
|
||||
} else if (msg.role === "user" && Array.isArray(msg.content)) {
|
||||
// Anthropic tool_result in user content blocks
|
||||
for (let i = 0; i < msg.content.length; i++) {
|
||||
if (msg.content[i]?.type === "tool_result") {
|
||||
msg.content[i].content = summaryContent;
|
||||
break;
|
||||
}
|
||||
}
|
||||
} else {
|
||||
if (Array.isArray(msg.content)) {
|
||||
msg.content = [{ type: "text", text: summaryContent }];
|
||||
} else {
|
||||
msg.content = summaryContent;
|
||||
}
|
||||
}
|
||||
msg._offloaded = true;
|
||||
}
|
||||
|
||||
// ─── Offload Map ─────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Build a lookup map from tool_call_id → OffloadEntry.
|
||||
*/
|
||||
export function buildOffloadMap(entries: OffloadEntry[]): Map<string, OffloadEntry> {
|
||||
const map = new Map<string, OffloadEntry>();
|
||||
for (const entry of entries) {
|
||||
if (entry.tool_call_id) {
|
||||
map.set(entry.tool_call_id, entry);
|
||||
}
|
||||
}
|
||||
return map;
|
||||
}
|
||||
|
||||
// ─── Token Estimation ────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Count CJK characters (CJK Unified Ideographs + Extension A + Compatibility).
|
||||
*/
|
||||
function countCjkChars(text: string): number {
|
||||
let n = 0;
|
||||
for (const ch of text) {
|
||||
const c = ch.codePointAt(0)!;
|
||||
if (
|
||||
(c >= 0x4e00 && c <= 0x9fff) ||
|
||||
(c >= 0x3400 && c <= 0x4dbf) ||
|
||||
(c >= 0xf900 && c <= 0xfaff)
|
||||
) {
|
||||
n++;
|
||||
}
|
||||
}
|
||||
return n;
|
||||
}
|
||||
|
||||
/**
|
||||
* Estimate tokens for a text string using CJK-aware heuristic.
|
||||
* CJK characters ≈ 1 token / 1.7 chars, non-CJK ≈ 1 token / 4 chars.
|
||||
* Aligned with plugin-side estimateL3MixedTokensHeuristic.
|
||||
*/
|
||||
function estimateTextTokens(text: string): number {
|
||||
const cjk = countCjkChars(text);
|
||||
const rest = Math.max(0, text.length - cjk);
|
||||
return Math.max(1, Math.ceil(cjk / 1.7 + rest / 4));
|
||||
}
|
||||
|
||||
/**
|
||||
* Estimate tokens for a message using CJK-aware heuristic (中文/1.7 + 非中文/4).
|
||||
*/
|
||||
export function estimateMessageTokens(msg: Message): number {
|
||||
const content = msg.content ?? msg.message?.content ?? "";
|
||||
let text: string;
|
||||
if (typeof content === "string") {
|
||||
text = content;
|
||||
} else if (Array.isArray(content)) {
|
||||
text = content.map((b: any) => (typeof b === "string" ? b : b?.text ?? JSON.stringify(b) ?? "")).join("");
|
||||
} else {
|
||||
text = JSON.stringify(content);
|
||||
}
|
||||
|
||||
// Include tool_calls arguments (OpenAI format)
|
||||
const toolCalls = (msg as any).tool_calls;
|
||||
if (toolCalls && Array.isArray(toolCalls)) {
|
||||
for (const tc of toolCalls) {
|
||||
if (tc.function?.name) text += tc.function.name;
|
||||
if (tc.function?.arguments) text += tc.function.arguments;
|
||||
}
|
||||
}
|
||||
|
||||
return estimateTextTokens(text);
|
||||
}
|
||||
|
||||
// ─── MMD Message Markers ─────────────────────────────────────────────────────
|
||||
|
||||
export const MMD_CONTEXT_MARKER = "_mmdContextMessage";
|
||||
export const MMD_INJECTION_MARKER = "_mmdInjection";
|
||||
|
||||
/**
|
||||
* Check if a message is a MMD-related message (should be preserved during compression).
|
||||
*/
|
||||
export function isMmdMessage(msg: Message): boolean {
|
||||
return !!msg[MMD_CONTEXT_MARKER] || !!msg[MMD_INJECTION_MARKER];
|
||||
}
|
||||
@@ -0,0 +1,430 @@
|
||||
/**
|
||||
* L3 MMD Injector — inject active/history MMD into messages.
|
||||
* Handles version dedup and insertion point calculation.
|
||||
*/
|
||||
import type { StorageAdapter } from "../../core/storage/adapter.js";
|
||||
import type { OffloadEntry, OffloadState } from "../types.js";
|
||||
import type { Message } from "./helpers.js";
|
||||
import { MMD_CONTEXT_MARKER, MMD_INJECTION_MARKER, buildOffloadMap } from "./helpers.js";
|
||||
|
||||
// ─── Active MMD Injection ────────────────────────────────────────────────────
|
||||
|
||||
export interface MmdInjectionResult {
|
||||
injectedCount: number;
|
||||
mmdTokensEstimate: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Inject the active MMD into messages. Performs version dedup.
|
||||
*/
|
||||
export async function injectActiveMmd(
|
||||
messages: Message[],
|
||||
state: OffloadState,
|
||||
storage: StorageAdapter,
|
||||
basePath: string,
|
||||
): Promise<MmdInjectionResult> {
|
||||
// Resolve the MMD file to inject: use activeMmdFile, or fallback to the latest .mmd
|
||||
let mmdFile = state.activeMmdFile;
|
||||
if (!mmdFile) {
|
||||
// Fallback: find the most recent .mmd file (sorted by prefix number, highest = latest)
|
||||
const allMmds = await storage.readdirNames(`${basePath}/mmds/`, ".mmd");
|
||||
if (allMmds.length > 0) {
|
||||
// Sort descending by prefix number (e.g., "006-current-task.mmd" → 6)
|
||||
allMmds.sort((a, b) => {
|
||||
const numA = parseInt(a.split("-")[0], 10) || 0;
|
||||
const numB = parseInt(b.split("-")[0], 10) || 0;
|
||||
return numB - numA;
|
||||
});
|
||||
// Pick the first non-empty one
|
||||
for (const candidate of allMmds) {
|
||||
const content = await storage.readFile(`${basePath}/mmds/${candidate}`);
|
||||
if (content && content.trim().length > 0) {
|
||||
mmdFile = candidate;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (!mmdFile) {
|
||||
removeMmdMessages(messages, "active");
|
||||
return { injectedCount: 0, mmdTokensEstimate: 0 };
|
||||
}
|
||||
|
||||
const mmdContent = await storage.readFile(`${basePath}/mmds/${mmdFile}`);
|
||||
if (!mmdContent) {
|
||||
removeMmdMessages(messages, "active");
|
||||
return { injectedCount: 0, mmdTokensEstimate: 0 };
|
||||
}
|
||||
|
||||
const newVersion = hashContent(mmdContent);
|
||||
|
||||
// Version dedup: check existing MMD message version and position
|
||||
const existingIdx = messages.findIndex((m) => m[MMD_CONTEXT_MARKER] === "active");
|
||||
if (existingIdx >= 0 && messages[existingIdx]._mmdVersion === newVersion) {
|
||||
// Content unchanged — but still need to ensure correct position
|
||||
// (aggressive compression may have shifted it)
|
||||
const correctIdx = findActiveMmdInsertionPoint(
|
||||
messages.filter((m) => m[MMD_CONTEXT_MARKER] !== "active"),
|
||||
);
|
||||
// Check if position is already correct (accounting for removal offset)
|
||||
const effectiveCorrectIdx = correctIdx <= existingIdx ? correctIdx : correctIdx;
|
||||
if (existingIdx === effectiveCorrectIdx) {
|
||||
return { injectedCount: 0, mmdTokensEstimate: 0 };
|
||||
}
|
||||
// Reposition: remove and re-insert at correct point
|
||||
const [mmdMsg] = messages.splice(existingIdx, 1);
|
||||
const newInsertIdx = findActiveMmdInsertionPoint(messages);
|
||||
messages.splice(newInsertIdx, 0, mmdMsg);
|
||||
return { injectedCount: 0, mmdTokensEstimate: 0 };
|
||||
}
|
||||
|
||||
// Remove old active MMD message
|
||||
removeMmdMessages(messages, "active");
|
||||
|
||||
// Build MMD text
|
||||
const mmdText = buildActiveMmdText(mmdFile, mmdContent);
|
||||
const mmdMsg: Message = {
|
||||
role: "user",
|
||||
content: mmdText,
|
||||
[MMD_CONTEXT_MARKER]: "active",
|
||||
_mmdVersion: newVersion,
|
||||
_mmdFilename: mmdFile,
|
||||
};
|
||||
|
||||
// Insert at calculated point
|
||||
const insertIdx = findActiveMmdInsertionPoint(messages);
|
||||
messages.splice(insertIdx, 0, mmdMsg);
|
||||
|
||||
return { injectedCount: 1, mmdTokensEstimate: Math.ceil(mmdContent.length / 4) };
|
||||
}
|
||||
|
||||
// ─── History MMD Injection ───────────────────────────────────────────────────
|
||||
|
||||
export interface HistoryMmdResult {
|
||||
injectedCount: number;
|
||||
mmdFiles: string[];
|
||||
totalTokensEstimate: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Inject history MMD files for entries that were deleted during aggressive compression.
|
||||
*/
|
||||
export async function injectHistoryMmds(
|
||||
messages: Message[],
|
||||
deletedIds: string[],
|
||||
entries: OffloadEntry[],
|
||||
state: OffloadState,
|
||||
storage: StorageAdapter,
|
||||
basePath: string,
|
||||
tokenBudget: number,
|
||||
): Promise<HistoryMmdResult> {
|
||||
if (deletedIds.length === 0) {
|
||||
return { injectedCount: 0, mmdFiles: [], totalTokensEstimate: 0 };
|
||||
}
|
||||
|
||||
// 1. Find MMD prefixes from deleted entries' node_ids
|
||||
const offloadMap = buildOffloadMap(entries);
|
||||
const mmdPrefixes = new Set<string>();
|
||||
for (const id of deletedIds) {
|
||||
const entry = offloadMap.get(id);
|
||||
if (entry?.node_id) {
|
||||
const prefix = entry.node_id.split("-")[0];
|
||||
if (prefix) mmdPrefixes.add(prefix);
|
||||
}
|
||||
}
|
||||
if (mmdPrefixes.size === 0) {
|
||||
return { injectedCount: 0, mmdFiles: [], totalTokensEstimate: 0 };
|
||||
}
|
||||
|
||||
// 2. Find matching history MMD files (exclude active)
|
||||
const allMmds = await storage.readdirNames(`${basePath}/mmds/`, ".mmd");
|
||||
const candidates = allMmds.filter((f) => {
|
||||
const prefix = f.split("-")[0];
|
||||
return mmdPrefixes.has(prefix) && f !== state.activeMmdFile;
|
||||
});
|
||||
if (candidates.length === 0) {
|
||||
return { injectedCount: 0, mmdFiles: [], totalTokensEstimate: 0 };
|
||||
}
|
||||
|
||||
// 3. Read and inject (respecting token budget), most recent first
|
||||
candidates.reverse();
|
||||
const injected: Message[] = [];
|
||||
const mmdFiles: string[] = [];
|
||||
let usedTokens = 0;
|
||||
|
||||
for (const filename of candidates) {
|
||||
const content = await storage.readFile(`${basePath}/mmds/${filename}`);
|
||||
if (!content) continue;
|
||||
|
||||
const text = buildHistoryMmdText(filename, content);
|
||||
const tokens = Math.ceil(text.length / 4);
|
||||
if (usedTokens + tokens > tokenBudget) continue;
|
||||
|
||||
injected.push({
|
||||
role: "user",
|
||||
content: text,
|
||||
[MMD_INJECTION_MARKER]: true,
|
||||
_mmdFilename: filename,
|
||||
});
|
||||
mmdFiles.push(filename);
|
||||
usedTokens += tokens;
|
||||
}
|
||||
|
||||
if (injected.length === 0) {
|
||||
return { injectedCount: 0, mmdFiles: [], totalTokensEstimate: 0 };
|
||||
}
|
||||
|
||||
// 4. Remove old history MMD injections and insert new ones
|
||||
removeExistingMmdInjections(messages);
|
||||
const insertIdx = findHistoryMmdInsertionPoint(messages);
|
||||
// Reverse back to chronological order (oldest first)
|
||||
injected.reverse();
|
||||
mmdFiles.reverse();
|
||||
messages.splice(insertIdx, 0, ...injected);
|
||||
|
||||
return { injectedCount: injected.length, mmdFiles, totalTokensEstimate: usedTokens };
|
||||
}
|
||||
|
||||
// ─── Insertion Point Calculation ─────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Find insertion point for active MMD.
|
||||
* Strategy: insert after the latest user message in the second half.
|
||||
* Guard: don't split tool_call / tool_result pairs.
|
||||
*/
|
||||
export function findActiveMmdInsertionPoint(messages: Message[]): number {
|
||||
if (messages.length <= 2) {
|
||||
const idx = Math.min(1, messages.length);
|
||||
return adjustForToolCallPair(messages, idx);
|
||||
}
|
||||
|
||||
let latestUserIdx = -1;
|
||||
for (let i = messages.length - 1; i >= 0; i--) {
|
||||
const role = messages[i].role ?? messages[i].message?.role ?? messages[i].type;
|
||||
if (role === "user" && !messages[i][MMD_CONTEXT_MARKER] && !messages[i][MMD_INJECTION_MARKER]) {
|
||||
latestUserIdx = i;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
let insertIdx: number;
|
||||
if (latestUserIdx >= 0) {
|
||||
// Insert relative to latest user message:
|
||||
// - If it's the last message → insert before it (user prompt stays last)
|
||||
// - Otherwise → insert after it (between user and tool loop, matching plugin behavior)
|
||||
if (latestUserIdx === messages.length - 1) {
|
||||
insertIdx = latestUserIdx;
|
||||
} else {
|
||||
insertIdx = latestUserIdx + 1;
|
||||
}
|
||||
} else {
|
||||
// No user message found — fallback: before the trailing tool loop
|
||||
let loopStart = messages.length;
|
||||
for (let i = messages.length - 1; i >= 0; i--) {
|
||||
const role = messages[i].role ?? messages[i].message?.role ?? messages[i].type;
|
||||
if (messages[i][MMD_CONTEXT_MARKER] || messages[i][MMD_INJECTION_MARKER]) continue;
|
||||
if (role === "toolResult" || role === "tool" || role === "assistant") {
|
||||
loopStart = i;
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
}
|
||||
const maxDistFromTail = 30;
|
||||
const minInsertIdx = Math.max(1, messages.length - maxDistFromTail);
|
||||
insertIdx = Math.max(loopStart, minInsertIdx);
|
||||
}
|
||||
|
||||
// Guard: don't insert between assistant(tool_use) and its tool_result
|
||||
insertIdx = adjustForToolCallPair(messages, insertIdx);
|
||||
|
||||
// Hard guard: never insert before system message
|
||||
if (insertIdx === 0 && messages.length > 0) {
|
||||
const firstRole = messages[0].role ?? messages[0].message?.role ?? messages[0].type;
|
||||
if (firstRole === "system") {
|
||||
insertIdx = 1;
|
||||
}
|
||||
}
|
||||
|
||||
return insertIdx;
|
||||
}
|
||||
|
||||
/**
|
||||
* Find insertion point for history MMD (before active MMD).
|
||||
*/
|
||||
export function findHistoryMmdInsertionPoint(messages: Message[]): number {
|
||||
for (let i = 0; i < messages.length; i++) {
|
||||
if (messages[i][MMD_CONTEXT_MARKER] === "active") return i;
|
||||
}
|
||||
return findActiveMmdInsertionPoint(messages);
|
||||
}
|
||||
|
||||
// ─── MMD Message Management ─────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Remove MMD messages by type ("active", "history", or all).
|
||||
*/
|
||||
export function removeMmdMessages(messages: Message[], type?: "active" | "history"): number {
|
||||
let removed = 0;
|
||||
for (let i = messages.length - 1; i >= 0; i--) {
|
||||
if (type === "active" && messages[i][MMD_CONTEXT_MARKER] === "active") {
|
||||
messages.splice(i, 1);
|
||||
removed++;
|
||||
} else if (type === "history" && messages[i][MMD_INJECTION_MARKER]) {
|
||||
messages.splice(i, 1);
|
||||
removed++;
|
||||
} else if (!type && (messages[i][MMD_CONTEXT_MARKER] || messages[i][MMD_INJECTION_MARKER])) {
|
||||
messages.splice(i, 1);
|
||||
removed++;
|
||||
}
|
||||
}
|
||||
return removed;
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove existing history MMD injection messages.
|
||||
*/
|
||||
export function removeExistingMmdInjections(messages: Message[]): number {
|
||||
let removed = 0;
|
||||
for (let i = messages.length - 1; i >= 0; i--) {
|
||||
if (messages[i][MMD_INJECTION_MARKER]) {
|
||||
messages.splice(i, 1);
|
||||
removed++;
|
||||
}
|
||||
}
|
||||
return removed;
|
||||
}
|
||||
|
||||
// ─── Text Builders ───────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Build active MMD injection text.
|
||||
*/
|
||||
export function buildActiveMmdText(filename: string, mmdContent: string): string {
|
||||
let taskGoal = "";
|
||||
const metaMatch = mmdContent.match(/^%%\{\s*(.*?)\s*\}%%/);
|
||||
if (metaMatch) {
|
||||
try {
|
||||
const meta = JSON.parse(`{${metaMatch[1]}}`);
|
||||
taskGoal = meta.taskGoal || "";
|
||||
} catch { /* ignore */ }
|
||||
}
|
||||
|
||||
return [
|
||||
`<current_task_context>`,
|
||||
`【当前活跃任务的mermaid流程图】这是你最近正在执行的任务的阶段性记录。`,
|
||||
taskGoal ? `**任务目标:** ${taskGoal}` : "",
|
||||
`**任务文件:** ${filename}`,
|
||||
"```mermaid",
|
||||
mmdContent,
|
||||
"```",
|
||||
`标记为 "doing" 的节点是近期焦点,"done" 的已完成。请参考此保持方向感,避免重复已完成的工作。`,
|
||||
`</current_task_context>`,
|
||||
]
|
||||
.filter((line) => line !== "")
|
||||
.join("\n");
|
||||
}
|
||||
|
||||
/**
|
||||
* Build history MMD injection text.
|
||||
*/
|
||||
export function buildHistoryMmdText(filename: string, mmdContent: string): string {
|
||||
let taskGoal = "";
|
||||
const metaMatch = mmdContent.match(/^%%\{\s*(.*?)\s*\}%%/);
|
||||
if (metaMatch) {
|
||||
try {
|
||||
const meta = JSON.parse(`{${metaMatch[1]}}`);
|
||||
taskGoal = meta.taskGoal || "";
|
||||
} catch { /* ignore */ }
|
||||
}
|
||||
|
||||
return [
|
||||
`<history_task_context>`,
|
||||
`【历史任务记录】以下是此前完成的任务的概要。`,
|
||||
taskGoal ? `**任务目标:** ${taskGoal}` : "",
|
||||
`**任务文件:** ${filename}`,
|
||||
"```mermaid",
|
||||
mmdContent,
|
||||
"```",
|
||||
`</history_task_context>`,
|
||||
]
|
||||
.filter((line) => line !== "")
|
||||
.join("\n");
|
||||
}
|
||||
|
||||
// ─── Internal Helpers ────────────────────────────────────────────────────────
|
||||
|
||||
function hasToolResultContent(msg: Message): boolean {
|
||||
const content = msg?.content;
|
||||
if (!Array.isArray(content)) return false;
|
||||
return content.some(
|
||||
(block: any) =>
|
||||
typeof block === "object" &&
|
||||
block !== null &&
|
||||
(block.type === "tool_result" || block.type === "toolResult"),
|
||||
);
|
||||
}
|
||||
|
||||
function hasToolUseContent(msg: Message): boolean {
|
||||
const content = msg?.content;
|
||||
if (!Array.isArray(content)) return false;
|
||||
return content.some(
|
||||
(block: any) =>
|
||||
typeof block === "object" &&
|
||||
block !== null &&
|
||||
(block.type === "tool_use" || block.type === "toolUse"),
|
||||
);
|
||||
}
|
||||
|
||||
function adjustForToolCallPair(messages: Message[], insertIdx: number): number {
|
||||
if (insertIdx <= 0 || insertIdx >= messages.length) return insertIdx;
|
||||
const msgAtIdx = messages[insertIdx];
|
||||
const role = msgAtIdx?.role ?? msgAtIdx?.message?.role ?? msgAtIdx?.type;
|
||||
|
||||
// Check both role-level (OpenAI: role="tool") and content-level (Anthropic: content[].type="tool_result")
|
||||
const isToolResult =
|
||||
role === "tool" ||
|
||||
role === "toolResult" ||
|
||||
role === "tool_result" ||
|
||||
(role === "user" && hasToolResultContent(msgAtIdx));
|
||||
|
||||
if (isToolResult) {
|
||||
// Walk back to before the assistant tool_use
|
||||
let i = insertIdx - 1;
|
||||
while (i >= 0) {
|
||||
const r = messages[i].role ?? messages[i].message?.role ?? messages[i].type;
|
||||
if (r === "assistant" && hasToolUseContent(messages[i])) {
|
||||
return i;
|
||||
}
|
||||
if (r === "assistant") {
|
||||
return i;
|
||||
}
|
||||
const prevIsToolResult =
|
||||
r === "tool" ||
|
||||
r === "toolResult" ||
|
||||
r === "tool_result" ||
|
||||
(r === "user" && hasToolResultContent(messages[i]));
|
||||
if (!prevIsToolResult) break;
|
||||
i--;
|
||||
}
|
||||
}
|
||||
|
||||
// Also check: don't insert right after an assistant with tool_use (before its tool_result)
|
||||
if (insertIdx > 0) {
|
||||
const prevMsg = messages[insertIdx - 1];
|
||||
const prevRole = prevMsg?.role ?? prevMsg?.message?.role ?? prevMsg?.type;
|
||||
if (prevRole === "assistant" && hasToolUseContent(prevMsg)) {
|
||||
return insertIdx - 1;
|
||||
}
|
||||
}
|
||||
|
||||
return insertIdx;
|
||||
}
|
||||
|
||||
function hashContent(s: string): string {
|
||||
let hash = 0;
|
||||
for (let i = 0; i < s.length; i++) {
|
||||
hash = ((hash << 5) - hash + s.charCodeAt(i)) | 0;
|
||||
}
|
||||
return Math.abs(hash).toString(36).padStart(6, "0").slice(0, 6);
|
||||
}
|
||||
@@ -0,0 +1,262 @@
|
||||
/**
|
||||
* Offload Ingest Handler — fast path: COS write + task enqueue.
|
||||
* Branch A: toolPairs only → write pending.jsonl, enqueue async L1
|
||||
* Branch B: recentMessages only (no toolPairs) → L1.5 path
|
||||
* Branch C: toolPairs + recentMessages → write pending + cache recentMessages, trigger L1 (skip L1.5)
|
||||
*/
|
||||
import type http from "node:http";
|
||||
import type { StorageAdapter } from "../core/storage/adapter.js";
|
||||
import type { IStateBackend, TaskPayload } from "../core/state/types.js";
|
||||
import type { OffloadExecutorConfig, OffloadState } from "./types.js";
|
||||
import { defaultOffloadState } from "./types.js";
|
||||
import { IngestRequestSchema } from "./schemas.js";
|
||||
import { serializeJsonl } from "./parsers/json-utils.js";
|
||||
import { buildOffloadBasePath } from "./session-utils.js";
|
||||
|
||||
// ─── Per-session in-process mutex for COS append serialization ───
|
||||
// Layer 1 (local): ensures at most one inflight appendFile per session within
|
||||
// the same Node.js process (no I/O overhead, zero-latency queue).
|
||||
// Layer 2 (distributed): stateBackend.acquireLock across multiple offload server
|
||||
// instances prevents concurrent AppendObject calls to the same COS key.
|
||||
const pendingMutexes = new Map<string, Promise<void>>();
|
||||
|
||||
async function withSessionMutex<T>(sessionKey: string, fn: () => Promise<T>): Promise<T> {
|
||||
const prev = pendingMutexes.get(sessionKey) ?? Promise.resolve();
|
||||
let resolve!: () => void;
|
||||
const next = new Promise<void>((r) => { resolve = r; });
|
||||
pendingMutexes.set(sessionKey, next);
|
||||
|
||||
try {
|
||||
await prev;
|
||||
return await fn();
|
||||
} finally {
|
||||
resolve();
|
||||
if (pendingMutexes.get(sessionKey) === next) {
|
||||
pendingMutexes.delete(sessionKey);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Max attempts to acquire the distributed lock. */
|
||||
const APPEND_LOCK_MAX_ATTEMPTS = 10;
|
||||
/** TTL for the distributed append lock (ms). Short-lived since append is fast. */
|
||||
const APPEND_LOCK_TTL_MS = 5000;
|
||||
/** Delay between lock acquisition retries (ms). Uses exponential backoff: base * 2^attempt. */
|
||||
const APPEND_LOCK_RETRY_BASE_MS = 50;
|
||||
|
||||
export interface IngestDeps {
|
||||
storage: StorageAdapter;
|
||||
stateBackend?: IStateBackend;
|
||||
config: OffloadExecutorConfig;
|
||||
logger: { info: (...args: unknown[]) => void; warn: (...args: unknown[]) => void };
|
||||
}
|
||||
|
||||
export async function handleIngest(
|
||||
req: http.IncomingMessage,
|
||||
res: http.ServerResponse,
|
||||
auth: { serviceId: string },
|
||||
deps: IngestDeps,
|
||||
requestId: string,
|
||||
parseJsonBody: <T>(req: http.IncomingMessage) => Promise<T>,
|
||||
sendJson: (res: http.ServerResponse, status: number, body: unknown) => void,
|
||||
successEnvelope: <T>(data: T, requestId: string) => unknown,
|
||||
errorEnvelope: (code: number, message: string, requestId: string) => unknown,
|
||||
): Promise<void> {
|
||||
const body = await parseJsonBody(req);
|
||||
const parsed = IngestRequestSchema.safeParse(body);
|
||||
if (!parsed.success) {
|
||||
sendJson(res, 400, errorEnvelope(400, parsed.error.message, requestId));
|
||||
return;
|
||||
}
|
||||
|
||||
const { session_id: sessionId, tool_pairs: toolPairs, prompt, recent_messages: recentMessages } = parsed.data;
|
||||
const { storage, stateBackend, config } = deps;
|
||||
const basePath = buildOffloadBasePath(sessionId);
|
||||
|
||||
// Build context string from structured prompt + recentMessages (for L1/L1.5)
|
||||
let contextText: string | undefined;
|
||||
if (prompt || (recentMessages && recentMessages.length > 0)) {
|
||||
const parts: string[] = [];
|
||||
if (recentMessages && recentMessages.length > 0) {
|
||||
parts.push("历史消息,可作为参考:");
|
||||
for (const m of recentMessages) {
|
||||
parts.push(`[${m.role === "user" ? "User" : "Assistant"}]: ${m.content}`);
|
||||
}
|
||||
}
|
||||
if (prompt) {
|
||||
parts.push(`\n最新user message:\n[User]: ${prompt}`);
|
||||
}
|
||||
contextText = parts.join("\n");
|
||||
}
|
||||
|
||||
// ─── Session skip filter (shared by L1 and L1.5) ───
|
||||
const INTERNAL_SESSION_RE = /memory-.*-session-\d+/;
|
||||
const shouldSkipSession = INTERNAL_SESSION_RE.test(sessionId) || sessionId.includes("subagent");
|
||||
|
||||
// ─── Branch L1: toolPairs non-empty → write pending + trigger L1 ───
|
||||
if (toolPairs.length > 0) {
|
||||
if (shouldSkipSession) {
|
||||
deps.logger.info(`[offload-server] ingest: L1 skipped (session=${sessionId})`);
|
||||
sendJson(res, 200, successEnvelope({}, requestId));
|
||||
return;
|
||||
}
|
||||
// Save context for L1 executor (if available)
|
||||
if (contextText) {
|
||||
await storage.writeFile(`${basePath}/recent-context.txt`, contextText);
|
||||
}
|
||||
|
||||
const pendingPath = `${basePath}/pending.jsonl`;
|
||||
// Map API snake_case → internal camelCase for JSONL storage
|
||||
const camelPairs = toolPairs.map((tp: Record<string, unknown>) => ({
|
||||
toolName: tp.tool_name,
|
||||
toolCallId: tp.tool_call_id,
|
||||
params: tp.params,
|
||||
result: tp.result,
|
||||
error: tp.error,
|
||||
timestamp: tp.timestamp,
|
||||
durationMs: tp.duration_ms,
|
||||
}));
|
||||
const lines = serializeJsonl(camelPairs);
|
||||
|
||||
// Two-layer serialization to prevent COS AppendPositionErr:
|
||||
// Layer 1 (local mutex): queues concurrent requests within this process.
|
||||
// Layer 2 (distributed lock via stateBackend): prevents races across server instances.
|
||||
// If the lock cannot be acquired, return 409 so the client retries (NOT proceed without lock).
|
||||
const lockAcquired = await withSessionMutex(pendingPath, async () => {
|
||||
const lockKey = `offload-pending:${auth.serviceId}:${sessionId}`;
|
||||
const lockOwner = requestId;
|
||||
let locked = false;
|
||||
|
||||
if (stateBackend) {
|
||||
for (let attempt = 0; attempt < APPEND_LOCK_MAX_ATTEMPTS; attempt++) {
|
||||
locked = await stateBackend.acquireLock(lockKey, lockOwner, APPEND_LOCK_TTL_MS);
|
||||
if (locked) break;
|
||||
// Exponential backoff: 50, 100, 200, 400, 800, 1600... capped at 2000ms
|
||||
const delay = Math.min(APPEND_LOCK_RETRY_BASE_MS * 2 ** attempt, 2000);
|
||||
await new Promise((r) => setTimeout(r, delay));
|
||||
}
|
||||
if (!locked) {
|
||||
deps.logger.warn(`[offload-server] ingest: append lock failed after ${APPEND_LOCK_MAX_ATTEMPTS} attempts (session=${sessionId}), returning 409`);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
await storage.appendFile(pendingPath, lines);
|
||||
|
||||
if (stateBackend) {
|
||||
const raw = await storage.readFile(pendingPath);
|
||||
const lineCount = raw ? raw.split("\n").filter(Boolean).length : 0;
|
||||
|
||||
if (lineCount >= config.forceTriggerThreshold) {
|
||||
const task: TaskPayload = {
|
||||
id: `offload-l1-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`,
|
||||
type: "offload-l1" as TaskPayload["type"],
|
||||
instanceId: auth.serviceId,
|
||||
sessionId,
|
||||
priority: 0,
|
||||
data: { sessionId, instanceId: auth.serviceId },
|
||||
createdAt: Date.now(),
|
||||
};
|
||||
await stateBackend.enqueueTask(task);
|
||||
deps.logger.info(`[offload-server] ingest: ${toolPairs.length} pairs, triggered=true (lines=${lineCount})`);
|
||||
} else {
|
||||
await stateBackend.setTimerIfEarlier(
|
||||
auth.serviceId,
|
||||
`offload-l1:${auth.serviceId}:${sessionId}`,
|
||||
Date.now() + config.pendingMaxAgeSeconds * 1000,
|
||||
);
|
||||
deps.logger.info(`[offload-server] ingest: ${toolPairs.length} pairs, lines=${lineCount}/${config.forceTriggerThreshold}, timer set`);
|
||||
}
|
||||
}
|
||||
return true;
|
||||
} finally {
|
||||
if (locked && stateBackend) {
|
||||
await stateBackend.releaseLock(lockKey, lockOwner);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
if (!lockAcquired) {
|
||||
sendJson(res, 409, errorEnvelope(409, "Concurrent write conflict, please retry", requestId));
|
||||
return;
|
||||
}
|
||||
|
||||
sendJson(res, 200, successEnvelope({}, requestId));
|
||||
return;
|
||||
}
|
||||
|
||||
// ─── Branch L1.5: toolPairs empty + prompt → task judgment ───
|
||||
// Skip L1.5 for inter-session messages, internal sessions, and system prompts
|
||||
const shouldSkipL15 = shouldSkipSession || !!(
|
||||
prompt && (
|
||||
prompt.startsWith("[Inter-session message]") ||
|
||||
prompt.startsWith("Pre-compaction")
|
||||
)
|
||||
);
|
||||
if (prompt && stateBackend && !shouldSkipL15) {
|
||||
const lockKey = `offload-state:${auth.serviceId}:${sessionId}`;
|
||||
const lockOwner = requestId;
|
||||
let locked = false;
|
||||
|
||||
// Use exponential backoff, same policy as L1 append lock.
|
||||
for (let attempt = 0; attempt < APPEND_LOCK_MAX_ATTEMPTS; attempt++) {
|
||||
locked = await stateBackend.acquireLock(lockKey, lockOwner, APPEND_LOCK_TTL_MS);
|
||||
if (locked) break;
|
||||
const delay = Math.min(APPEND_LOCK_RETRY_BASE_MS * 2 ** attempt, 2000);
|
||||
await new Promise((r) => setTimeout(r, delay));
|
||||
}
|
||||
|
||||
if (!locked) {
|
||||
// Do NOT proceed without the lock: a concurrent writer would overwrite state.json
|
||||
// and silently drop this boundary, causing the L1.5 executor to skip the task
|
||||
// (findIndex returns -1 → return without error).
|
||||
deps.logger.warn(`[offload-server] ingest: L1.5 state lock failed after ${APPEND_LOCK_MAX_ATTEMPTS} attempts (session=${sessionId}), returning 409`);
|
||||
sendJson(res, 409, errorEnvelope(409, "Concurrent write conflict, please retry", requestId));
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
const state = await readState(storage, basePath);
|
||||
const boundaryTimestamp = new Date().toISOString();
|
||||
state.boundaries.push({
|
||||
targetMmd: "_pending",
|
||||
timestamp: boundaryTimestamp,
|
||||
});
|
||||
await writeState(storage, basePath, state);
|
||||
|
||||
const task: TaskPayload = {
|
||||
id: `offload-l15-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`,
|
||||
type: "offload-l15" as TaskPayload["type"],
|
||||
instanceId: auth.serviceId,
|
||||
sessionId,
|
||||
priority: 0,
|
||||
data: { sessionId, recentMessages: contextText, boundaryTimestamp, instanceId: auth.serviceId },
|
||||
createdAt: Date.now(),
|
||||
};
|
||||
await stateBackend.enqueueTask(task);
|
||||
deps.logger.info(`[offload-server] ingest: L1.5 triggered, boundaryTs=${boundaryTimestamp}, prompt=${prompt.slice(0, 200)}`);
|
||||
} finally {
|
||||
await stateBackend.releaseLock(lockKey, lockOwner);
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Fast return ───
|
||||
sendJson(res, 200, successEnvelope({}, requestId));
|
||||
}
|
||||
|
||||
// ─── State helpers (duplicated intentionally to keep ingest-handler self-contained) ──
|
||||
|
||||
async function readState(storage: StorageAdapter, basePath: string): Promise<OffloadState> {
|
||||
const raw = await storage.readFile(`${basePath}/state.json`);
|
||||
if (!raw) return defaultOffloadState();
|
||||
try {
|
||||
return { ...defaultOffloadState(), ...JSON.parse(raw) };
|
||||
} catch {
|
||||
return defaultOffloadState();
|
||||
}
|
||||
}
|
||||
|
||||
async function writeState(storage: StorageAdapter, basePath: string, state: OffloadState): Promise<void> {
|
||||
await storage.writeFile(`${basePath}/state.json`, JSON.stringify(state));
|
||||
}
|
||||
@@ -0,0 +1,90 @@
|
||||
/**
|
||||
* Offload MMD Query Handler — returns MMD files.
|
||||
* Body params: sessionId (required), limit (optional, default=all)
|
||||
* When limit=1, only returns the active MMD.
|
||||
*/
|
||||
import type http from "node:http";
|
||||
import type { StorageAdapter } from "../core/storage/adapter.js";
|
||||
import type { OffloadState } from "./types.js";
|
||||
import { createHash } from "node:crypto";
|
||||
import { buildOffloadBasePath } from "./session-utils.js";
|
||||
|
||||
function hashContent(content: string): string {
|
||||
return createHash("md5").update(content).digest("hex").slice(0, 12);
|
||||
}
|
||||
|
||||
export async function handleMmdQuery(
|
||||
_req: http.IncomingMessage,
|
||||
res: http.ServerResponse,
|
||||
_auth: { serviceId: string },
|
||||
storage: StorageAdapter,
|
||||
requestId: string,
|
||||
sendJson: (res: http.ServerResponse, status: number, body: unknown) => void,
|
||||
successEnvelope: <T>(data: T, requestId: string) => unknown,
|
||||
errorEnvelope: (code: number, message: string, requestId: string) => unknown,
|
||||
sessionId: string,
|
||||
limit?: number,
|
||||
): Promise<void> {
|
||||
if (!sessionId) {
|
||||
sendJson(res, 400, errorEnvelope(400, "missing sessionId", requestId));
|
||||
return;
|
||||
}
|
||||
|
||||
const basePath = buildOffloadBasePath(sessionId);
|
||||
|
||||
// Read state
|
||||
const stateRaw = await storage.readFile(`${basePath}/state.json`);
|
||||
let state: Partial<OffloadState> = {};
|
||||
if (stateRaw) {
|
||||
try {
|
||||
state = JSON.parse(stateRaw);
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
}
|
||||
|
||||
const mmdsPrefix = `${basePath}/mmds/`;
|
||||
const mmds: Array<{
|
||||
filename: string;
|
||||
content: string;
|
||||
version: string;
|
||||
}> = [];
|
||||
|
||||
if (limit === 1 && state.activeMmdFile) {
|
||||
// Fast path: only return the active MMD
|
||||
const content = await storage.readFile(`${mmdsPrefix}${state.activeMmdFile}`) ?? "";
|
||||
mmds.push({
|
||||
filename: state.activeMmdFile,
|
||||
content,
|
||||
version: content ? hashContent(content) : "",
|
||||
});
|
||||
} else {
|
||||
// Return all (or up to limit)
|
||||
const mmdFiles = await storage.readdirNames(mmdsPrefix, ".mmd");
|
||||
const filesToRead = limit && limit > 0 ? mmdFiles.slice(0, limit) : mmdFiles;
|
||||
|
||||
const readResults = await Promise.all(
|
||||
filesToRead.map(async (filename) => {
|
||||
const content = await storage.readFile(`${mmdsPrefix}${filename}`) ?? "";
|
||||
return {
|
||||
filename,
|
||||
content,
|
||||
version: content ? hashContent(content) : "",
|
||||
};
|
||||
}),
|
||||
);
|
||||
mmds.push(...readResults);
|
||||
}
|
||||
|
||||
sendJson(
|
||||
res,
|
||||
200,
|
||||
successEnvelope(
|
||||
{
|
||||
mmds,
|
||||
currentMmd: state.activeMmdFile ?? null,
|
||||
},
|
||||
requestId,
|
||||
),
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,745 @@
|
||||
/**
|
||||
* Offload Task Executor — async L1/L1.5/L2 execution via PipelineWorker.
|
||||
*/
|
||||
import type { StorageAdapter } from "../core/storage/adapter.js";
|
||||
import type { IStateBackend, TaskPayload } from "../core/state/types.js";
|
||||
import type {
|
||||
OffloadEntry,
|
||||
OffloadState,
|
||||
OffloadExecutorConfig,
|
||||
L2ParsedResponse,
|
||||
MmdMeta,
|
||||
ToolPair,
|
||||
} from "./types.js";
|
||||
import { defaultOffloadState, defaultOffloadConfig } from "./types.js";
|
||||
import { parseJsonl, serializeJsonl } from "./parsers/json-utils.js";
|
||||
import { parseL1Response } from "./parsers/l1-parser.js";
|
||||
import { parseL15Response } from "./parsers/l15-parser.js";
|
||||
import { parseL2Response } from "./parsers/l2-parser.js";
|
||||
import { L1_SYSTEM_PROMPT, buildL1UserPrompt } from "./prompts/l1-prompt.js";
|
||||
import { L15_SYSTEM_PROMPT, buildL15UserPrompt } from "./prompts/l15-prompt.js";
|
||||
import { L2_SYSTEM_PROMPT, buildL2UserPrompt } from "./prompts/l2-prompt.js";
|
||||
import { handleTaskTransition, extractMmdMeta } from "./task-transition.js";
|
||||
import { buildOffloadBasePath } from "./session-utils.js";
|
||||
import { traceServerModelIo, traceServerTaskDecision } from "./opik-tracer.js";
|
||||
|
||||
// ─── LLM Client Interface ────────────────────────────────────────────────────
|
||||
|
||||
export interface LlmClient {
|
||||
chat(params: {
|
||||
model: string;
|
||||
messages: Array<{ role: "system" | "user"; content: string }>;
|
||||
temperature: number;
|
||||
max_tokens: number;
|
||||
timeoutMs?: number;
|
||||
}): Promise<string>;
|
||||
}
|
||||
|
||||
// ─── Logger Interface ────────────────────────────────────────────────────────
|
||||
|
||||
export interface ExecutorLogger {
|
||||
info: (...args: unknown[]) => void;
|
||||
warn: (...args: unknown[]) => void;
|
||||
error: (...args: unknown[]) => void;
|
||||
}
|
||||
|
||||
// ─── Executor Deps ───────────────────────────────────────────────────────────
|
||||
|
||||
export interface OffloadExecutorDeps {
|
||||
resolveStorage: (instanceId: string) => Promise<StorageAdapter | undefined>;
|
||||
llmClient: LlmClient;
|
||||
stateBackend: IStateBackend;
|
||||
config?: OffloadExecutorConfig;
|
||||
logger: ExecutorLogger;
|
||||
}
|
||||
|
||||
// ─── Executor Class ──────────────────────────────────────────────────────────
|
||||
|
||||
export class OffloadTaskExecutor {
|
||||
private deps: OffloadExecutorDeps;
|
||||
private config: OffloadExecutorConfig;
|
||||
|
||||
constructor(deps: OffloadExecutorDeps) {
|
||||
this.deps = deps;
|
||||
this.config = deps.config ?? defaultOffloadConfig();
|
||||
}
|
||||
|
||||
// ═══════════════════════════════════════════════════════════════════════════
|
||||
// L1: Summarize pending ToolPairs → OffloadEntry[]
|
||||
// ═══════════════════════════════════════════════════════════════════════════
|
||||
|
||||
async executeOffloadL1(task: TaskPayload, _signal?: AbortSignal): Promise<void> {
|
||||
const startMs = Date.now();
|
||||
const sessionId = this.extractSessionId(task);
|
||||
if (!sessionId) return;
|
||||
|
||||
const storage = await this.resolveStorageOrThrow(task.instanceId);
|
||||
const basePath = buildOffloadBasePath(sessionId);
|
||||
const pendingPath = `${basePath}/pending.jsonl`;
|
||||
|
||||
// 1. Claim pending via rename (atomic: new ingest appends to fresh pending.jsonl)
|
||||
const processingPath = `${basePath}/pending-processing-${task.id}.jsonl`;
|
||||
const pendingRaw = await storage.readFile(pendingPath);
|
||||
if (!pendingRaw || !pendingRaw.trim()) return;
|
||||
|
||||
try {
|
||||
await storage.rename(pendingPath, processingPath);
|
||||
} catch {
|
||||
// rename failed (file gone — another L1 claimed it, or doesn't exist)
|
||||
return;
|
||||
}
|
||||
|
||||
// Re-read from processing file (rename succeeded, this is our exclusive copy)
|
||||
const claimedRaw = await storage.readFile(processingPath);
|
||||
if (!claimedRaw || !claimedRaw.trim()) {
|
||||
await storage.unlink(processingPath);
|
||||
return;
|
||||
}
|
||||
|
||||
let toolPairs = parseJsonl<Record<string, unknown>>(claimedRaw, (line, err) => {
|
||||
this.deps.logger.warn(`[offload-server] L1: bad JSONL line in pending: ${line}`, err);
|
||||
});
|
||||
if (toolPairs.length === 0) {
|
||||
await storage.unlink(processingPath);
|
||||
return;
|
||||
}
|
||||
|
||||
// 2. Dedup: remove toolPairs whose toolCallId already exists in entries.jsonl
|
||||
const entriesPath = `${basePath}/entries.jsonl`;
|
||||
const existingEntriesRaw = await storage.readFile(entriesPath);
|
||||
if (existingEntriesRaw) {
|
||||
const existingIds = new Set<string>();
|
||||
for (const e of parseJsonl<OffloadEntry>(existingEntriesRaw)) {
|
||||
if (e.tool_call_id) existingIds.add(e.tool_call_id);
|
||||
}
|
||||
const before = toolPairs.length;
|
||||
toolPairs = toolPairs.filter((tp) => {
|
||||
const id = tp.toolCallId as string;
|
||||
return !id || !existingIds.has(id);
|
||||
});
|
||||
if (toolPairs.length < before) {
|
||||
this.deps.logger.info(`[offload-server] L1: dedup removed ${before - toolPairs.length}/${before} duplicate toolPairs`);
|
||||
}
|
||||
if (toolPairs.length === 0) {
|
||||
await storage.unlink(processingPath);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Build prompt & call LLM
|
||||
const recentContext = await storage.readFile(`${basePath}/recent-context.txt`) ?? "";
|
||||
const userPrompt = buildL1UserPrompt(recentContext, toolPairs as unknown as ToolPair[]);
|
||||
|
||||
let newEntries: OffloadEntry[];
|
||||
const l1LlmStart = Date.now();
|
||||
let l1RawResponse: string | undefined;
|
||||
try {
|
||||
const response = await this.deps.llmClient.chat({
|
||||
model: this.config.l1Model,
|
||||
messages: [
|
||||
{ role: "system", content: L1_SYSTEM_PROMPT },
|
||||
{ role: "user", content: userPrompt },
|
||||
],
|
||||
temperature: this.config.l1Temperature,
|
||||
max_tokens: this.config.l1MaxTokens,
|
||||
timeoutMs: this.config.l1TimeoutMs,
|
||||
});
|
||||
l1RawResponse = response;
|
||||
newEntries = parseL1Response(response);
|
||||
traceServerModelIo({
|
||||
sessionId,
|
||||
stage: "L1",
|
||||
model: this.config.l1Model,
|
||||
systemPrompt: L1_SYSTEM_PROMPT,
|
||||
userPrompt,
|
||||
responseContent: response,
|
||||
status: "ok",
|
||||
durationMs: Date.now() - l1LlmStart,
|
||||
logger: this.deps.logger,
|
||||
});
|
||||
} catch (err) {
|
||||
traceServerModelIo({
|
||||
sessionId,
|
||||
stage: "L1",
|
||||
model: this.config.l1Model,
|
||||
systemPrompt: L1_SYSTEM_PROMPT,
|
||||
userPrompt,
|
||||
responseContent: l1RawResponse ?? "",
|
||||
status: "error",
|
||||
errorMessage: String(err),
|
||||
durationMs: Date.now() - l1LlmStart,
|
||||
logger: this.deps.logger,
|
||||
});
|
||||
this.deps.logger.error(`[offload-server] L1 LLM failed:`, err);
|
||||
throw err; // Let Worker retry (processing file remains for re-claim)
|
||||
}
|
||||
|
||||
// 4. Ensure all toolPairs are covered — fallback for missing entries
|
||||
const parsedIds = new Set(newEntries.map((e) => e.tool_call_id));
|
||||
// Build toolCallId → original timestamp map for reliable boundary matching
|
||||
const tpTimestampMap = new Map<string, string>();
|
||||
for (const tp of toolPairs) {
|
||||
const id = tp.toolCallId as string;
|
||||
if (id && tp.timestamp) tpTimestampMap.set(id, tp.timestamp as string);
|
||||
}
|
||||
// Overwrite entry timestamps with original tool pair timestamps (don't trust LLM output)
|
||||
for (const entry of newEntries) {
|
||||
const origTs = tpTimestampMap.get(entry.tool_call_id);
|
||||
if (origTs) entry.timestamp = origTs;
|
||||
}
|
||||
for (const tp of toolPairs) {
|
||||
const id = tp.toolCallId as string;
|
||||
if (id && !parsedIds.has(id)) {
|
||||
newEntries.push({
|
||||
tool_call_id: id,
|
||||
tool_call: (tp.toolName as string) ?? "",
|
||||
summary: "[L1 parse incomplete]",
|
||||
timestamp: (tp.timestamp as string) ?? new Date().toISOString(),
|
||||
score: 2,
|
||||
node_id: null,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// 5. Write refs: store original tool result content for each pair
|
||||
for (const tp of toolPairs) {
|
||||
const id = tp.toolCallId as string;
|
||||
if (!id) continue;
|
||||
const result = tp.result ?? tp.error ?? "";
|
||||
const resultStr = typeof result === "string" ? result : JSON.stringify(result, null, 2);
|
||||
if (!resultStr || resultStr.length < 20) continue; // skip trivially short results
|
||||
const toolName = (tp.toolName as string) ?? "unknown";
|
||||
const timestamp = (tp.timestamp as string) ?? new Date().toISOString();
|
||||
const header = `# Tool Result: ${toolName}\n\n**tool_call_id:** ${id}\n**Timestamp:** ${timestamp}\n\n---\n\n`;
|
||||
const refPath = `${basePath}/refs/${id}.md`;
|
||||
await storage.writeFile(refPath, header + resultStr);
|
||||
// Set result_ref on the matching entry (full relative path for tdai_read_cos)
|
||||
const entry = newEntries.find((e) => e.tool_call_id === id);
|
||||
if (entry) entry.result_ref = `${basePath}/refs/${id}.md`;
|
||||
}
|
||||
|
||||
// 6. Write entries (atomic append) + delete processing file
|
||||
await storage.appendFile(entriesPath, serializeJsonl(newEntries));
|
||||
await storage.unlink(processingPath);
|
||||
|
||||
// 7. Check L2 trigger: group null entries by boundary targetMmd
|
||||
const state = await this.readState(storage, basePath);
|
||||
const allEntriesRaw = await storage.readFile(entriesPath);
|
||||
if (allEntriesRaw && state.boundaries.length > 0) {
|
||||
const allEntries = parseJsonl<OffloadEntry>(allEntriesRaw, (line, err) => {
|
||||
this.deps.logger.warn(`[offload-server] L1: bad JSONL line in entries: ${line}`, err);
|
||||
});
|
||||
const nodeMapping = await this.readNodeMapping(storage, basePath);
|
||||
|
||||
// Group null entries by resolved targetMmd
|
||||
const nullByMmd = new Map<string, number>();
|
||||
for (const e of allEntries) {
|
||||
if (this.getEffectiveNodeId(e, nodeMapping) !== null || !e.timestamp) continue;
|
||||
const boundary = this.findBoundaryByTimestamp(state.boundaries, e.timestamp);
|
||||
if (!boundary) continue; // no boundary → ignore
|
||||
if (boundary.targetMmd === "_pending" || !boundary.targetMmd) continue; // pending → ignore
|
||||
nullByMmd.set(boundary.targetMmd, (nullByMmd.get(boundary.targetMmd) ?? 0) + 1);
|
||||
}
|
||||
|
||||
// Trigger L2 for each MMD that reached threshold
|
||||
for (const [mmdFile, count] of nullByMmd) {
|
||||
if (count >= this.config.l2NullThreshold) {
|
||||
// Threshold met: short delay (1s) so concurrent L1s merge into one L2
|
||||
await this.deps.stateBackend.setTimerIfEarlier(
|
||||
task.instanceId,
|
||||
`offload-l2:${task.instanceId}:${sessionId}:${mmdFile}`,
|
||||
Date.now() + 1_000,
|
||||
);
|
||||
this.deps.logger.info(`[offload-server] L2 timer set (fast, mmd=${mmdFile}, nullCount=${count})`);
|
||||
} else if (count > 0) {
|
||||
await this.deps.stateBackend.setTimerIfEarlier(
|
||||
task.instanceId,
|
||||
`offload-l2:${task.instanceId}:${sessionId}:${mmdFile}`,
|
||||
Date.now() + 30_000,
|
||||
);
|
||||
this.deps.logger.info(`[offload-server] L2 timer set (mmd=${mmdFile}, nullCount=${count})`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
this.deps.logger.info(
|
||||
`[offload-server] L1 complete: ${newEntries.length} entries produced (${Date.now() - startMs}ms)`,
|
||||
);
|
||||
}
|
||||
|
||||
// ═══════════════════════════════════════════════════════════════════════════
|
||||
// L1.5: Task judgment — triggered by user new message
|
||||
// ═══════════════════════════════════════════════════════════════════════════
|
||||
|
||||
async executeOffloadL15(task: TaskPayload, _signal?: AbortSignal): Promise<void> {
|
||||
const startMs = Date.now();
|
||||
const { sessionId, recentMessages, boundaryTimestamp } = task.data as {
|
||||
sessionId: string;
|
||||
recentMessages?: string;
|
||||
boundaryTimestamp: string;
|
||||
};
|
||||
const storage = await this.resolveStorageOrThrow(task.instanceId);
|
||||
const basePath = buildOffloadBasePath(sessionId);
|
||||
|
||||
// ─── Phase 1: Read-only snapshot + LLM call (NO LOCK) ─────────────────
|
||||
// Multiple L1.5 tasks can execute this phase concurrently. Each operates
|
||||
// on its own boundary and the LLM call (3-10s) does not block others.
|
||||
|
||||
// 1. Read current active MMD (snapshot for prompt building)
|
||||
const preState = await this.readState(storage, basePath);
|
||||
let currentMmd: { filename: string; content: string } | null = null;
|
||||
if (preState.activeMmdFile) {
|
||||
const content = await storage.readFile(`${basePath}/mmds/${preState.activeMmdFile}`);
|
||||
if (content) {
|
||||
currentMmd = { filename: preState.activeMmdFile, content };
|
||||
}
|
||||
}
|
||||
|
||||
// 2. List available MMDs → extract metas
|
||||
const mmdsPrefix = `${basePath}/mmds/`;
|
||||
const mmdFiles = await storage.readdirNames(mmdsPrefix, ".mmd");
|
||||
const metas: MmdMeta[] = [];
|
||||
for (const f of mmdFiles) {
|
||||
const content = await storage.readFile(`${mmdsPrefix}${f}`);
|
||||
if (content) {
|
||||
metas.push(extractMmdMeta(f, content));
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Build prompt & call LLM (no lock held — this is the expensive part)
|
||||
const userPrompt = buildL15UserPrompt(
|
||||
recentMessages ?? "",
|
||||
currentMmd,
|
||||
metas,
|
||||
);
|
||||
|
||||
let judgment;
|
||||
let rawResponse: string | undefined;
|
||||
const l15LlmStart = Date.now();
|
||||
try {
|
||||
const response = await this.deps.llmClient.chat({
|
||||
model: this.config.l15Model,
|
||||
messages: [
|
||||
{ role: "system", content: L15_SYSTEM_PROMPT },
|
||||
{ role: "user", content: userPrompt },
|
||||
],
|
||||
temperature: this.config.l15Temperature,
|
||||
max_tokens: this.config.l15MaxTokens,
|
||||
timeoutMs: this.config.l15TimeoutMs,
|
||||
});
|
||||
rawResponse = response;
|
||||
judgment = parseL15Response(response);
|
||||
traceServerModelIo({
|
||||
sessionId,
|
||||
stage: "L1.5",
|
||||
model: this.config.l15Model,
|
||||
systemPrompt: L15_SYSTEM_PROMPT,
|
||||
userPrompt,
|
||||
responseContent: response,
|
||||
status: "ok",
|
||||
durationMs: Date.now() - l15LlmStart,
|
||||
logger: this.deps.logger,
|
||||
});
|
||||
} catch (err) {
|
||||
traceServerModelIo({
|
||||
sessionId,
|
||||
stage: "L1.5",
|
||||
model: this.config.l15Model,
|
||||
systemPrompt: L15_SYSTEM_PROMPT,
|
||||
userPrompt,
|
||||
responseContent: rawResponse ?? "",
|
||||
status: "error",
|
||||
errorMessage: String(err),
|
||||
durationMs: Date.now() - l15LlmStart,
|
||||
logger: this.deps.logger,
|
||||
});
|
||||
this.deps.logger.error(`[offload-server] L1.5 LLM failed:`, err);
|
||||
throw err;
|
||||
}
|
||||
|
||||
if (!judgment) {
|
||||
this.deps.logger.warn(`[offload-server] L1.5: null response (parse failed), raw=${rawResponse?.slice(0, 500) ?? "(empty)"}`);
|
||||
return;
|
||||
}
|
||||
|
||||
this.deps.logger.info(
|
||||
`[offload-server] L1.5: completed=${judgment.taskCompleted}, long=${judgment.isLongTask}, cont=${judgment.isContinuation}, label=${judgment.newTaskLabel ?? "none"} (LLM=${Date.now() - l15LlmStart}ms)`,
|
||||
);
|
||||
|
||||
// Trace L1.5 decision
|
||||
traceServerTaskDecision({
|
||||
sessionId,
|
||||
judgment: judgment as unknown as Record<string, unknown>,
|
||||
durationMs: Date.now() - startMs,
|
||||
logger: this.deps.logger,
|
||||
});
|
||||
|
||||
// ─── Phase 2: Write phase (SHORT LOCK) ────────────────────────────────
|
||||
// Only lock during state.json read-modify-write. Lock TTL is short (10s)
|
||||
// because this phase only does file I/O (no LLM calls).
|
||||
const lockKey = `offload-state:${task.instanceId}:${sessionId}`;
|
||||
const lockOwner = task.id;
|
||||
let locked = false;
|
||||
for (let attempt = 0; attempt < 10; attempt++) {
|
||||
locked = await this.deps.stateBackend.acquireLock(lockKey, lockOwner, 10_000);
|
||||
if (locked) break;
|
||||
await new Promise((r) => setTimeout(r, 100 + attempt * 50));
|
||||
}
|
||||
if (!locked) {
|
||||
this.deps.logger.warn(`[offload-server] L1.5: failed to acquire write lock after 10 attempts (ts=${boundaryTimestamp}), skipping`);
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
// Re-read state (fresh, may have been modified by another L1.5 that finished earlier)
|
||||
const state = await this.readState(storage, basePath);
|
||||
|
||||
// CAS check: find boundary by timestamp
|
||||
const boundaryIdx = state.boundaries.findIndex((b) => b.timestamp === boundaryTimestamp);
|
||||
if (boundaryIdx < 0) {
|
||||
this.deps.logger.warn(`[offload-server] L1.5: boundary not found for ts=${boundaryTimestamp}, skipping`);
|
||||
return;
|
||||
}
|
||||
|
||||
// CAS check: already backfilled by another concurrent L1.5? Skip.
|
||||
if (state.boundaries[boundaryIdx].targetMmd !== "_pending") {
|
||||
this.deps.logger.info(`[offload-server] L1.5: boundary[ts=${boundaryTimestamp}] already backfilled, skipping`);
|
||||
return;
|
||||
}
|
||||
|
||||
// Apply task transition
|
||||
await handleTaskTransition(state, judgment, storage, basePath);
|
||||
|
||||
// Backfill boundary targetMmd
|
||||
state.boundaries[boundaryIdx].targetMmd = state.activeMmdFile;
|
||||
|
||||
// Update state
|
||||
await this.writeState(storage, basePath, state);
|
||||
|
||||
// Check L2 trigger: scan ALL null entries grouped by targetMmd.
|
||||
// This covers the case where L1 completed before L1.5 backfill —
|
||||
// L1 skipped entries with _pending boundaries, so L1.5 must pick them up.
|
||||
const entriesRaw = await storage.readFile(`${basePath}/entries.jsonl`);
|
||||
if (entriesRaw) {
|
||||
const allEntries = parseJsonl<OffloadEntry>(entriesRaw);
|
||||
const l15NodeMapping = await this.readNodeMapping(storage, basePath);
|
||||
|
||||
// Group null entries by their resolved targetMmd (using all boundaries, not just current)
|
||||
const nullByMmd = new Map<string, number>();
|
||||
for (const e of allEntries) {
|
||||
if (this.getEffectiveNodeId(e, l15NodeMapping) !== null || !e.timestamp) continue;
|
||||
const boundary = this.findBoundaryByTimestamp(state.boundaries, e.timestamp);
|
||||
if (!boundary) continue;
|
||||
if (boundary.targetMmd === "_pending" || !boundary.targetMmd) continue;
|
||||
nullByMmd.set(boundary.targetMmd, (nullByMmd.get(boundary.targetMmd) ?? 0) + 1);
|
||||
}
|
||||
|
||||
for (const [mmdFile, nullCount] of nullByMmd) {
|
||||
if (nullCount >= this.config.l2NullThreshold) {
|
||||
await this.deps.stateBackend.setTimerIfEarlier(
|
||||
task.instanceId,
|
||||
`offload-l2:${task.instanceId}:${sessionId}:${mmdFile}`,
|
||||
Date.now() + 1_000,
|
||||
);
|
||||
this.deps.logger.info(`[offload-server] L1.5: L2 timer set (fast, mmd=${mmdFile}, nullCount=${nullCount})`);
|
||||
} else if (nullCount > 0) {
|
||||
await this.deps.stateBackend.setTimerIfEarlier(
|
||||
task.instanceId,
|
||||
`offload-l2:${task.instanceId}:${sessionId}:${mmdFile}`,
|
||||
Date.now() + 30_000,
|
||||
);
|
||||
this.deps.logger.info(`[offload-server] L1.5: L2 timer set after backfill (mmd=${mmdFile}, nullCount=${nullCount})`);
|
||||
}
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
if (locked) {
|
||||
await this.deps.stateBackend.releaseLock(lockKey, lockOwner);
|
||||
}
|
||||
}
|
||||
|
||||
this.deps.logger.info(
|
||||
`[offload-server] L1.5 complete: boundary[ts=${boundaryTimestamp}] (total=${Date.now() - startMs}ms)`,
|
||||
);
|
||||
}
|
||||
|
||||
// ═══════════════════════════════════════════════════════════════════════════
|
||||
// L2: MMD generation / update + node_id backfill
|
||||
// ═══════════════════════════════════════════════════════════════════════════
|
||||
|
||||
async executeOffloadL2(task: TaskPayload, _signal?: AbortSignal): Promise<void> {
|
||||
const startMs = Date.now();
|
||||
const sessionId = this.extractSessionId(task);
|
||||
if (!sessionId) return;
|
||||
|
||||
let targetMmdFile: string | undefined;
|
||||
const data = task.data as Record<string, unknown>;
|
||||
targetMmdFile = data.targetMmdFile as string | undefined;
|
||||
|
||||
// Extract targetMmdFile from timer member if available
|
||||
// Format: "offload-l2:{instanceId}:{sessionId}:{mmdFile}" → extract last segment ending in .mmd
|
||||
if (!targetMmdFile && data.timerMember) {
|
||||
const timerMember = data.timerMember as string;
|
||||
const mmdMatch = timerMember.match(/(\d+-[^:]+\.mmd)$/);
|
||||
if (mmdMatch) {
|
||||
targetMmdFile = mmdMatch[1];
|
||||
}
|
||||
}
|
||||
|
||||
const storage = await this.resolveStorageOrThrow(task.instanceId);
|
||||
const basePath = buildOffloadBasePath(sessionId);
|
||||
const state = await this.readState(storage, basePath);
|
||||
|
||||
// Resolve targetMmdFile: from task data, timer member, or state (fallback)
|
||||
if (!targetMmdFile) {
|
||||
targetMmdFile = state.activeMmdFile ?? undefined;
|
||||
}
|
||||
if (!targetMmdFile) return;
|
||||
|
||||
// 1. Read all entries + node mapping
|
||||
const entriesRaw = await storage.readFile(`${basePath}/entries.jsonl`);
|
||||
if (!entriesRaw) return;
|
||||
const allEntries = parseJsonl<OffloadEntry>(entriesRaw);
|
||||
const nodeMapping = await this.readNodeMapping(storage, basePath);
|
||||
|
||||
// 2. Filter: entries belonging to targetMmdFile with null node_id (after join)
|
||||
const relevantEntries = allEntries.filter((e) => {
|
||||
if (this.getEffectiveNodeId(e, nodeMapping) !== null) return false;
|
||||
if (!e.timestamp) return false;
|
||||
const boundary = this.findBoundaryByTimestamp(state.boundaries, e.timestamp);
|
||||
return boundary?.targetMmd === targetMmdFile;
|
||||
});
|
||||
|
||||
if (relevantEntries.length === 0) return;
|
||||
|
||||
// Guard: if oldest null entry is older than 10 minutes, give up and assign fallback node_id
|
||||
// to prevent infinite L2 retries when LLM consistently fails to map certain entries.
|
||||
const L2_MAX_AGE_MS = 10 * 60 * 1000;
|
||||
const oldestTs = relevantEntries.reduce((min, e) => {
|
||||
const t = new Date(e.timestamp).getTime();
|
||||
return t < min ? t : min;
|
||||
}, Infinity);
|
||||
if (Date.now() - oldestTs > L2_MAX_AGE_MS) {
|
||||
this.deps.logger.warn(
|
||||
`[offload-server] L2: ${relevantEntries.length} entries exceeded max age (10min), assigning fallback node_id`,
|
||||
);
|
||||
const fallbackMappings = relevantEntries.map((e) => ({
|
||||
tool_call_id: e.tool_call_id,
|
||||
node_id: `${targetMmdFile!.replace(/\.mmd$/, "")}-orphan`,
|
||||
}));
|
||||
const mappingPath = `${basePath}/node-mapping.jsonl`;
|
||||
await storage.appendFile(mappingPath, serializeJsonl(fallbackMappings));
|
||||
return;
|
||||
}
|
||||
|
||||
// 3. Read existing MMD
|
||||
const mmdPath = `${basePath}/mmds/${targetMmdFile}`;
|
||||
const existingMmd = await storage.readFile(mmdPath);
|
||||
|
||||
// 4. Build prompt (include recent context for better MMD generation)
|
||||
const recentHistory = await storage.readFile(`${basePath}/recent-context.txt`) ?? null;
|
||||
const taskLabel = targetMmdFile.replace(/^\d+-/, "").replace(/\.mmd$/, "") || "task";
|
||||
const prefixMatch = targetMmdFile.match(/^(\d+)-/);
|
||||
const mmdPrefix = prefixMatch ? prefixMatch[1] : "000";
|
||||
const charCount = existingMmd?.length ?? 0;
|
||||
|
||||
const userPrompt = buildL2UserPrompt({
|
||||
existingMmd: existingMmd || null,
|
||||
entries: relevantEntries,
|
||||
recentHistory,
|
||||
taskLabel,
|
||||
mmdPrefix,
|
||||
charCount,
|
||||
});
|
||||
|
||||
// 5. Call LLM
|
||||
let result: L2ParsedResponse | null;
|
||||
let rawL2Response: string | undefined;
|
||||
const l2LlmStart = Date.now();
|
||||
try {
|
||||
const response = await this.deps.llmClient.chat({
|
||||
model: this.config.l2Model,
|
||||
messages: [
|
||||
{ role: "system", content: L2_SYSTEM_PROMPT },
|
||||
{ role: "user", content: userPrompt },
|
||||
],
|
||||
temperature: this.config.l2Temperature,
|
||||
max_tokens: this.config.l2MaxTokens,
|
||||
timeoutMs: this.config.l2TimeoutMs,
|
||||
});
|
||||
rawL2Response = response;
|
||||
result = parseL2Response(response);
|
||||
traceServerModelIo({
|
||||
sessionId: sessionId!,
|
||||
stage: "L2",
|
||||
model: this.config.l2Model,
|
||||
systemPrompt: L2_SYSTEM_PROMPT,
|
||||
userPrompt,
|
||||
responseContent: response,
|
||||
status: "ok",
|
||||
durationMs: Date.now() - l2LlmStart,
|
||||
logger: this.deps.logger,
|
||||
});
|
||||
} catch (err) {
|
||||
traceServerModelIo({
|
||||
sessionId: sessionId!,
|
||||
stage: "L2",
|
||||
model: this.config.l2Model,
|
||||
systemPrompt: L2_SYSTEM_PROMPT,
|
||||
userPrompt,
|
||||
responseContent: rawL2Response ?? "",
|
||||
status: "error",
|
||||
errorMessage: String(err),
|
||||
durationMs: Date.now() - l2LlmStart,
|
||||
logger: this.deps.logger,
|
||||
});
|
||||
this.deps.logger.error(`[offload-server] L2 LLM failed:`, err);
|
||||
throw err;
|
||||
}
|
||||
|
||||
if (!result) {
|
||||
this.deps.logger.warn(`[offload-server] L2: parse failed, raw=${rawL2Response?.slice(0, 500) ?? "(empty)"}`);
|
||||
// Schedule retry so remaining null entries are not orphaned
|
||||
await this.deps.stateBackend.setTimerIfEarlier(
|
||||
task.instanceId,
|
||||
`offload-l2:${task.instanceId}:${sessionId}:${targetMmdFile}`,
|
||||
Date.now() + 30_000,
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
// 6. Apply MMD update
|
||||
const updatedMmd = this.applyL2Result(existingMmd ?? "", result);
|
||||
await storage.writeFile(mmdPath, updatedMmd);
|
||||
|
||||
// 7. Backfill: write node mappings to separate file (avoids overwriting entries.jsonl)
|
||||
const mappingEntries = Object.entries(result.nodeMapping).map(([toolCallId, nodeId]) => ({
|
||||
tool_call_id: toolCallId,
|
||||
node_id: nodeId,
|
||||
}));
|
||||
if (mappingEntries.length > 0) {
|
||||
const mappingPath = `${basePath}/node-mapping.jsonl`;
|
||||
await storage.appendFile(mappingPath, serializeJsonl(mappingEntries));
|
||||
}
|
||||
|
||||
// 8. Check remaining null entries for this MMD — re-read node-mapping for accurate count
|
||||
// (another L2 may have concurrently written mappings, or LLM may have missed some)
|
||||
const freshNodeMapping = await this.readNodeMapping(storage, basePath);
|
||||
const remainingNull = relevantEntries.filter(
|
||||
(e) => !freshNodeMapping.has(e.tool_call_id) && !result!.nodeMapping[e.tool_call_id],
|
||||
).length;
|
||||
if (remainingNull > 0) {
|
||||
await this.deps.stateBackend.setTimerIfEarlier(
|
||||
task.instanceId,
|
||||
`offload-l2:${task.instanceId}:${sessionId}:${targetMmdFile}`,
|
||||
Date.now() + 30_000,
|
||||
);
|
||||
this.deps.logger.info(`[offload-server] L2 retry timer set (mmd=${targetMmdFile}, remainingNull=${remainingNull})`);
|
||||
}
|
||||
|
||||
this.deps.logger.info(
|
||||
`[offload-server] L2 complete: ${Object.keys(result.nodeMapping).length} entries mapped, action=${result.fileAction} (${Date.now() - startMs}ms)`,
|
||||
);
|
||||
}
|
||||
|
||||
// ═══════════════════════════════════════════════════════════════════════════
|
||||
// Helpers
|
||||
// ═══════════════════════════════════════════════════════════════════════════
|
||||
|
||||
/**
|
||||
* Extract sessionId from task data or task.sessionId.
|
||||
*/
|
||||
private extractSessionId(task: TaskPayload): string | undefined {
|
||||
const data = task.data as Record<string, unknown> | undefined;
|
||||
// Prefer explicit sessionId in data
|
||||
if (data?.sessionId && typeof data.sessionId === "string") {
|
||||
return data.sessionId;
|
||||
}
|
||||
// Fallback: task-level sessionId
|
||||
if (task.sessionId) {
|
||||
return task.sessionId;
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
private async resolveStorageOrThrow(instanceId: string): Promise<StorageAdapter> {
|
||||
const storage = await this.deps.resolveStorage(instanceId);
|
||||
if (!storage) throw new Error(`Storage unavailable for instance ${instanceId}`);
|
||||
return storage;
|
||||
}
|
||||
|
||||
private async readState(storage: StorageAdapter, basePath: string): Promise<OffloadState> {
|
||||
const raw = await storage.readFile(`${basePath}/state.json`);
|
||||
if (!raw) return defaultOffloadState();
|
||||
try {
|
||||
return { ...defaultOffloadState(), ...JSON.parse(raw) };
|
||||
} catch {
|
||||
return defaultOffloadState();
|
||||
}
|
||||
}
|
||||
|
||||
private async writeState(storage: StorageAdapter, basePath: string, state: OffloadState): Promise<void> {
|
||||
await storage.writeFile(`${basePath}/state.json`, JSON.stringify(state));
|
||||
}
|
||||
|
||||
private findBoundaryByTimestamp(
|
||||
boundaries: OffloadState["boundaries"],
|
||||
entryTimestamp: string,
|
||||
): OffloadState["boundaries"][number] | null {
|
||||
if (boundaries.length === 0) return null;
|
||||
// Find the last boundary whose timestamp <= entryTimestamp
|
||||
let result: OffloadState["boundaries"][number] | null = null;
|
||||
for (const b of boundaries) {
|
||||
if (b.timestamp <= entryTimestamp) result = b;
|
||||
else break;
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private applyL2Result(existingMmd: string, result: L2ParsedResponse): string {
|
||||
if (result.fileAction === "write" && result.mmdContent) {
|
||||
return result.mmdContent;
|
||||
}
|
||||
|
||||
if (result.fileAction === "replace" && result.replaceBlocks?.length) {
|
||||
const lines = existingMmd.split("\n");
|
||||
// Sort blocks by startLine descending to avoid offset issues
|
||||
const sorted = [...result.replaceBlocks]
|
||||
.filter((block) => block.startLine >= 1 && block.endLine >= block.startLine && block.startLine <= lines.length)
|
||||
.sort((a, b) => b.startLine - a.startLine);
|
||||
for (const block of sorted) {
|
||||
const start = block.startLine - 1; // 0-based
|
||||
const end = Math.min(block.endLine, lines.length); // clamp endLine
|
||||
const deleteCount = end - block.startLine + 1;
|
||||
const newLines = block.content.split("\n");
|
||||
lines.splice(start, deleteCount, ...newLines);
|
||||
}
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
return existingMmd;
|
||||
}
|
||||
|
||||
/**
|
||||
* Read node-mapping.jsonl and build a tool_call_id → node_id map.
|
||||
*/
|
||||
private async readNodeMapping(storage: StorageAdapter, basePath: string): Promise<Map<string, string>> {
|
||||
const raw = await storage.readFile(`${basePath}/node-mapping.jsonl`);
|
||||
const map = new Map<string, string>();
|
||||
if (!raw) return map;
|
||||
const lines = parseJsonl<{ tool_call_id: string; node_id: string }>(raw);
|
||||
for (const line of lines) {
|
||||
if (line.tool_call_id && line.node_id) {
|
||||
map.set(line.tool_call_id, line.node_id);
|
||||
}
|
||||
}
|
||||
return map;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get effective node_id for an entry: check node-mapping first, then entry's own field.
|
||||
*/
|
||||
private getEffectiveNodeId(entry: OffloadEntry, nodeMapping: Map<string, string>): string | null {
|
||||
return nodeMapping.get(entry.tool_call_id) ?? entry.node_id;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,336 @@
|
||||
/**
|
||||
* Opik observability tracer for offload server.
|
||||
* Wraps the opik npm package with graceful degradation when not installed.
|
||||
* Configuration is read from environment variables (no OpenClaw plugin config).
|
||||
*/
|
||||
|
||||
// Opik client types (minimal shape to avoid hard dependency)
|
||||
interface OpikClient {
|
||||
trace(params: Record<string, unknown>): OpikTrace;
|
||||
flush(): Promise<void>;
|
||||
}
|
||||
interface OpikTrace {
|
||||
update(params: Record<string, unknown>): void;
|
||||
end(): void;
|
||||
span(params: Record<string, unknown>): OpikSpan;
|
||||
}
|
||||
interface OpikSpan {
|
||||
update(params: Record<string, unknown>): void;
|
||||
end(): void;
|
||||
}
|
||||
|
||||
// ─── Module State ────────────────────────────────────────────────────────────
|
||||
|
||||
let client: OpikClient | null = null;
|
||||
let tracerEnabled = false;
|
||||
let tracerInitTried = false;
|
||||
|
||||
// ─── Helpers ─────────────────────────────────────────────────────────────────
|
||||
|
||||
function extractLayerTag(stage: string): string {
|
||||
const match = stage.match(/^(L\d+(?:\.\d+)?)/i);
|
||||
if (!match) return "Lx-unknown";
|
||||
return match[1].toUpperCase();
|
||||
}
|
||||
|
||||
function durationBucketTag(ms: number): string {
|
||||
if (typeof ms !== "number" || ms < 0) return "duration:unknown";
|
||||
if (ms < 1000) return "duration:<1s";
|
||||
if (ms < 5000) return "duration:1-5s";
|
||||
if (ms < 15000) return "duration:5-15s";
|
||||
if (ms < 30000) return "duration:15-30s";
|
||||
return "duration:>30s";
|
||||
}
|
||||
|
||||
function formatDuration(ms: number): string {
|
||||
if (typeof ms !== "number" || ms < 0) return "?";
|
||||
if (ms < 1000) return `${Math.round(ms)}ms`;
|
||||
return `${(ms / 1000).toFixed(2)}s`;
|
||||
}
|
||||
|
||||
// ─── Logger Interface ────────────────────────────────────────────────────────
|
||||
|
||||
export interface TracerLogger {
|
||||
info: (message: string, ...args: unknown[]) => void;
|
||||
warn: (message: string, ...args: unknown[]) => void;
|
||||
debug?: (message: string, ...args: unknown[]) => void;
|
||||
}
|
||||
|
||||
// ─── Init ────────────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Initialize the offload server Opik tracer.
|
||||
* Reads config from environment variables:
|
||||
* OPIK_ENABLED — set to "true" to enable (default: disabled)
|
||||
* OPIK_URL_OVERRIDE — Opik server URL
|
||||
* OPIK_API_KEY — API key
|
||||
* OPIK_WORKSPACE — workspace name (default: "default")
|
||||
* OPIK_PROJECT_NAME — project name (default: "openclaw-offload-server")
|
||||
*/
|
||||
export async function initServerOpikTracer(logger: TracerLogger): Promise<void> {
|
||||
if (tracerInitTried) return;
|
||||
tracerInitTried = true;
|
||||
try {
|
||||
const enabled = process.env.OPIK_ENABLED === "true";
|
||||
if (!enabled) {
|
||||
logger.debug?.("[offload-server] Opik tracer disabled (OPIK_ENABLED != true)");
|
||||
return;
|
||||
}
|
||||
|
||||
const apiUrl = process.env.OPIK_URL_OVERRIDE;
|
||||
const apiKey = process.env.OPIK_API_KEY;
|
||||
const workspaceName = process.env.OPIK_WORKSPACE ?? "default";
|
||||
const projectName = process.env.OPIK_PROJECT_NAME ?? "openclaw-offload-server";
|
||||
|
||||
// Dynamic import — graceful when opik is not installed
|
||||
let OpikConstructor: new (params: Record<string, unknown>) => OpikClient;
|
||||
let disableOpikLogger: (() => void) | undefined;
|
||||
try {
|
||||
const opikModule = await import("opik") as {
|
||||
Opik: new (params: Record<string, unknown>) => OpikClient;
|
||||
disableLogger?: () => void;
|
||||
setLoggerLevel?: (level: string) => void;
|
||||
};
|
||||
OpikConstructor = opikModule.Opik;
|
||||
disableOpikLogger = opikModule.disableLogger;
|
||||
} catch {
|
||||
logger.debug?.("[offload-server] opik package not available, tracer disabled");
|
||||
return;
|
||||
}
|
||||
|
||||
// Suppress opik internal logs (flush messages, ANSI color noise)
|
||||
if (disableOpikLogger) {
|
||||
disableOpikLogger();
|
||||
}
|
||||
|
||||
client = new OpikConstructor({
|
||||
...(apiKey ? { apiKey } : {}),
|
||||
...(apiUrl ? { apiUrl } : {}),
|
||||
workspaceName,
|
||||
projectName,
|
||||
});
|
||||
tracerEnabled = true;
|
||||
logger.info(
|
||||
`[offload-server] Opik tracer enabled: project=${projectName}, workspace=${workspaceName}`,
|
||||
);
|
||||
} catch (err) {
|
||||
tracerEnabled = false;
|
||||
client = null;
|
||||
logger.warn(`[offload-server] Opik tracer init failed: ${String(err)}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if the tracer is enabled and ready to trace.
|
||||
*/
|
||||
export function isTracerEnabled(): boolean {
|
||||
return tracerEnabled && client !== null;
|
||||
}
|
||||
|
||||
// ─── Trace: Model I/O (L1/L1.5/L2 LLM calls) ───────────────────────────────
|
||||
|
||||
/**
|
||||
* Trace LLM model I/O for offload server L1/L1.5/L2 stages.
|
||||
*/
|
||||
export function traceServerModelIo(params: {
|
||||
sessionId: string;
|
||||
stage: string;
|
||||
model: string;
|
||||
systemPrompt: string;
|
||||
userPrompt: string;
|
||||
responseContent: string;
|
||||
usage?: { promptTokens?: number; completionTokens?: number; totalTokens?: number };
|
||||
status: "ok" | "error";
|
||||
errorMessage?: string;
|
||||
durationMs: number;
|
||||
logger?: TracerLogger;
|
||||
}): void {
|
||||
if (!tracerEnabled || !client) return;
|
||||
try {
|
||||
const layerTag = extractLayerTag(params.stage);
|
||||
const threadId = params.sessionId || `offload-server-${Date.now()}`;
|
||||
const dur = params.durationMs;
|
||||
const durStr = formatDuration(dur);
|
||||
const durBucket = durationBucketTag(dur);
|
||||
const skTag = `session:${params.sessionId || "unknown"}`;
|
||||
|
||||
const trace = client.trace({
|
||||
name: `${params.model} · offload-server · ${params.stage} · ${durStr}`,
|
||||
threadId,
|
||||
metadata: {
|
||||
plugin: "openclaw-offload-server",
|
||||
category: "llm",
|
||||
stage: params.stage,
|
||||
layer: layerTag,
|
||||
model: params.model,
|
||||
sessionId: params.sessionId,
|
||||
durationMs: dur,
|
||||
duration: durStr,
|
||||
},
|
||||
tags: ["offload-server", "llm", layerTag, durBucket, skTag],
|
||||
});
|
||||
|
||||
const span = trace.span({
|
||||
name: `${params.model} · ${params.stage} · ${durStr}`,
|
||||
type: "llm",
|
||||
model: params.model,
|
||||
input: {
|
||||
systemPrompt: params.systemPrompt,
|
||||
userPrompt: params.userPrompt,
|
||||
},
|
||||
metadata: {
|
||||
stage: params.stage,
|
||||
layer: layerTag,
|
||||
sessionId: params.sessionId,
|
||||
durationMs: dur,
|
||||
duration: durStr,
|
||||
},
|
||||
});
|
||||
|
||||
span.update({
|
||||
output: {
|
||||
responseContent: params.responseContent,
|
||||
usage: params.usage,
|
||||
durationMs: dur,
|
||||
duration: durStr,
|
||||
error: params.errorMessage,
|
||||
},
|
||||
metadata: {
|
||||
status: params.status,
|
||||
durationMs: dur,
|
||||
},
|
||||
});
|
||||
span.end();
|
||||
trace.end();
|
||||
void client.flush().catch(() => undefined);
|
||||
} catch (err) {
|
||||
params.logger?.warn?.(`[offload-server] Opik model I/O trace failed: ${String(err)}`);
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Trace: Compaction Decision ──────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Trace compaction (L3) decision and results.
|
||||
*/
|
||||
export function traceServerCompaction(params: {
|
||||
sessionId: string;
|
||||
level: string;
|
||||
ratio: number;
|
||||
contextWindow: number;
|
||||
totalTokensBefore: number;
|
||||
totalTokensAfter: number;
|
||||
originalMsgCount: number;
|
||||
compactedMsgCount: number;
|
||||
report: Record<string, unknown>;
|
||||
messages: unknown[];
|
||||
durationMs: number;
|
||||
logger?: TracerLogger;
|
||||
}): void {
|
||||
if (!tracerEnabled || !client) return;
|
||||
try {
|
||||
const threadId = params.sessionId || `offload-server-${Date.now()}`;
|
||||
const dur = params.durationMs;
|
||||
const durStr = formatDuration(dur);
|
||||
const durBucket = durationBucketTag(dur);
|
||||
const skTag = `session:${params.sessionId || "unknown"}`;
|
||||
|
||||
const trace = client.trace({
|
||||
name: `compaction · L3 · ${params.level} · ${durStr} [${params.sessionId}]`,
|
||||
threadId,
|
||||
input: {
|
||||
level: params.level,
|
||||
ratio: params.ratio,
|
||||
contextWindow: params.contextWindow,
|
||||
totalTokensBefore: params.totalTokensBefore,
|
||||
originalMsgCount: params.originalMsgCount,
|
||||
},
|
||||
metadata: {
|
||||
plugin: "openclaw-offload-server",
|
||||
category: "compaction",
|
||||
stage: "L3",
|
||||
layer: "L3",
|
||||
level: params.level,
|
||||
sessionId: params.sessionId,
|
||||
durationMs: dur,
|
||||
duration: durStr,
|
||||
},
|
||||
tags: ["offload-server", "compaction", "L3", `level:${params.level}`, durBucket, skTag],
|
||||
});
|
||||
|
||||
// Serialize messages for full snapshot
|
||||
const serializedMessages = params.messages.map((msg: any, i: number) => {
|
||||
const role = msg.role ?? "unknown";
|
||||
const content = typeof msg.content === "string"
|
||||
? msg.content
|
||||
: Array.isArray(msg.content)
|
||||
? msg.content.map((c: any) => {
|
||||
if (c.type === "text") return c.text;
|
||||
if (c.type === "tool_use") return `[tool_use: ${c.name} id=${c.id}]`;
|
||||
if (c.type === "tool_result") return `[tool_result: id=${c.tool_use_id} content=${typeof c.content === "string" ? c.content.slice(0, 500) : JSON.stringify(c.content).slice(0, 500)}]`;
|
||||
return `[${c.type ?? "unknown"}]`;
|
||||
}).join("\n")
|
||||
: "";
|
||||
return { i, role, content, ...(msg._mmdContextMessage ? { mmdCtx: true } : {}), ...(msg._offloaded ? { offloaded: true } : {}) };
|
||||
});
|
||||
|
||||
trace.update({
|
||||
output: {
|
||||
totalTokensAfter: params.totalTokensAfter,
|
||||
compactedMsgCount: params.compactedMsgCount,
|
||||
tokenReduction: params.totalTokensBefore - params.totalTokensAfter,
|
||||
report: params.report,
|
||||
messages: serializedMessages,
|
||||
},
|
||||
});
|
||||
trace.end();
|
||||
void client.flush().catch(() => undefined);
|
||||
} catch (err) {
|
||||
params.logger?.warn?.(`[offload-server] Opik compaction trace failed: ${String(err)}`);
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Trace: Task Decision (L1.5 judgment result) ─────────────────────────────
|
||||
|
||||
/**
|
||||
* Trace L1.5 task judgment decision.
|
||||
*/
|
||||
export function traceServerTaskDecision(params: {
|
||||
sessionId: string;
|
||||
judgment: Record<string, unknown>;
|
||||
durationMs: number;
|
||||
logger?: TracerLogger;
|
||||
}): void {
|
||||
if (!tracerEnabled || !client) return;
|
||||
try {
|
||||
const threadId = params.sessionId || `offload-server-${Date.now()}`;
|
||||
const dur = params.durationMs;
|
||||
const durStr = formatDuration(dur);
|
||||
const skTag = `session:${params.sessionId || "unknown"}`;
|
||||
|
||||
const trace = client.trace({
|
||||
name: `task-decision · L1.5 · ${durStr} [${params.sessionId}]`,
|
||||
threadId,
|
||||
input: {
|
||||
stage: "L1.5",
|
||||
sessionId: params.sessionId,
|
||||
},
|
||||
metadata: {
|
||||
plugin: "openclaw-offload-server",
|
||||
category: "decision",
|
||||
stage: "L1.5",
|
||||
layer: "L1.5",
|
||||
sessionId: params.sessionId,
|
||||
durationMs: dur,
|
||||
duration: durStr,
|
||||
},
|
||||
tags: ["offload-server", "decision", "L1.5", skTag],
|
||||
});
|
||||
|
||||
trace.update({ output: params.judgment });
|
||||
trace.end();
|
||||
void client.flush().catch(() => undefined);
|
||||
} catch (err) {
|
||||
params.logger?.warn?.(`[offload-server] Opik task decision trace failed: ${String(err)}`);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,92 @@
|
||||
/**
|
||||
* JSON / Mermaid extraction utilities for LLM response parsing.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Extract JSON from raw LLM output. Tolerates markdown fences, extra text.
|
||||
*/
|
||||
export function extractJson<T>(raw: string): T | null {
|
||||
if (!raw || typeof raw !== "string") return null;
|
||||
|
||||
const trimmed = raw.trim();
|
||||
|
||||
// 1. Direct parse
|
||||
try {
|
||||
return JSON.parse(trimmed) as T;
|
||||
} catch {
|
||||
// continue
|
||||
}
|
||||
|
||||
// 2. Extract from ```json ... ``` fence
|
||||
const jsonFenceMatch = trimmed.match(/```(?:json)?\s*\n?([\s\S]*?)```/);
|
||||
if (jsonFenceMatch) {
|
||||
try {
|
||||
return JSON.parse(jsonFenceMatch[1].trim()) as T;
|
||||
} catch {
|
||||
// continue
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Extract first { ... } (greedy last })
|
||||
const objStart = trimmed.indexOf("{");
|
||||
const objEnd = trimmed.lastIndexOf("}");
|
||||
if (objStart !== -1 && objEnd > objStart) {
|
||||
try {
|
||||
return JSON.parse(trimmed.slice(objStart, objEnd + 1)) as T;
|
||||
} catch {
|
||||
// continue
|
||||
}
|
||||
}
|
||||
|
||||
// 4. Extract first [ ... ] (greedy last ])
|
||||
const arrStart = trimmed.indexOf("[");
|
||||
const arrEnd = trimmed.lastIndexOf("]");
|
||||
if (arrStart !== -1 && arrEnd > arrStart) {
|
||||
try {
|
||||
return JSON.parse(trimmed.slice(arrStart, arrEnd + 1)) as T;
|
||||
} catch {
|
||||
// continue
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract mermaid content from a ```mermaid ... ``` code fence.
|
||||
*/
|
||||
export function extractMermaidFromFence(raw: string): string | null {
|
||||
if (!raw) return null;
|
||||
const match = raw.match(/```mermaid\s*\n?([\s\S]*?)```/);
|
||||
return match ? match[1].trim() : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse JSONL (newline-delimited JSON) string into array.
|
||||
* Corrupted lines are silently skipped (logged via optional callback).
|
||||
*/
|
||||
export function parseJsonl<T>(
|
||||
content: string,
|
||||
onBadLine?: (line: string, error: unknown) => void,
|
||||
): T[] {
|
||||
if (!content || !content.trim()) return [];
|
||||
const results: T[] = [];
|
||||
for (const line of content.split("\n")) {
|
||||
const trimmed = line.trim();
|
||||
if (trimmed.length === 0) continue;
|
||||
try {
|
||||
results.push(JSON.parse(trimmed) as T);
|
||||
} catch (err) {
|
||||
onBadLine?.(trimmed, err);
|
||||
}
|
||||
}
|
||||
return results;
|
||||
}
|
||||
|
||||
/**
|
||||
* Serialize array to JSONL string (trailing newline).
|
||||
*/
|
||||
export function serializeJsonl<T>(items: T[]): string {
|
||||
if (items.length === 0) return "";
|
||||
return items.map((item) => JSON.stringify(item)).join("\n") + "\n";
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
/**
|
||||
* L1 Response Parser — extracts OffloadEntry[] from LLM output.
|
||||
*/
|
||||
import { extractJson } from "./json-utils.js";
|
||||
import type { OffloadEntry } from "../types.js";
|
||||
|
||||
interface RawL1Entry {
|
||||
tool_call?: string;
|
||||
summary?: string;
|
||||
tool_call_id?: string;
|
||||
timestamp?: string;
|
||||
score?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse L1 LLM response into OffloadEntry array.
|
||||
* Tolerant of markdown wrapping, missing fields, etc.
|
||||
*/
|
||||
export function parseL1Response(raw: string): OffloadEntry[] {
|
||||
const parsed = extractJson<RawL1Entry[]>(raw);
|
||||
if (!parsed || !Array.isArray(parsed)) return [];
|
||||
|
||||
const entries: OffloadEntry[] = [];
|
||||
for (const item of parsed) {
|
||||
if (!item || typeof item !== "object") continue;
|
||||
const toolCallId = item.tool_call_id ?? "";
|
||||
if (!toolCallId) continue;
|
||||
|
||||
entries.push({
|
||||
tool_call_id: toolCallId,
|
||||
tool_call: item.tool_call ?? "",
|
||||
summary: item.summary ?? "",
|
||||
timestamp: item.timestamp ?? "",
|
||||
score: typeof item.score === "number" ? Math.min(10, Math.max(0, item.score)) : 5,
|
||||
node_id: null,
|
||||
seq: -1,
|
||||
});
|
||||
}
|
||||
|
||||
return entries;
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
/**
|
||||
* L1.5 Response Parser — extracts TaskJudgment from LLM output.
|
||||
*/
|
||||
import { extractJson } from "./json-utils.js";
|
||||
import type { TaskJudgment } from "../types.js";
|
||||
|
||||
interface RawL15Response {
|
||||
taskCompleted?: boolean | null;
|
||||
isContinuation?: boolean | null;
|
||||
isLongTask?: boolean | null;
|
||||
continuationMmdFile?: string | null;
|
||||
newTaskLabel?: string | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse L1.5 LLM response into TaskJudgment.
|
||||
* Returns null if completely unparseable or all-null (LLM unavailable).
|
||||
*/
|
||||
export function parseL15Response(raw: string): TaskJudgment | null {
|
||||
const parsed = extractJson<RawL15Response>(raw);
|
||||
if (!parsed || typeof parsed !== "object") return null;
|
||||
|
||||
if (
|
||||
parsed.taskCompleted == null &&
|
||||
parsed.isContinuation == null &&
|
||||
parsed.isLongTask == null
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return {
|
||||
taskCompleted: toBool(parsed.taskCompleted),
|
||||
isContinuation: toBool(parsed.isContinuation),
|
||||
isLongTask: toBool(parsed.isLongTask),
|
||||
continuationMmdFile:
|
||||
typeof parsed.continuationMmdFile === "string" && isSafeFilename(parsed.continuationMmdFile)
|
||||
? parsed.continuationMmdFile
|
||||
: undefined,
|
||||
newTaskLabel:
|
||||
typeof parsed.newTaskLabel === "string" ? parsed.newTaskLabel : undefined,
|
||||
};
|
||||
}
|
||||
|
||||
/** Safely coerce LLM value to boolean, handling string "false"/"0". */
|
||||
function toBool(value: unknown): boolean {
|
||||
if (typeof value === "string") {
|
||||
return value.toLowerCase() !== "false" && value !== "0" && value !== "";
|
||||
}
|
||||
return Boolean(value);
|
||||
}
|
||||
|
||||
/** Validate that a filename is safe (no path traversal or special chars). */
|
||||
function isSafeFilename(name: string): boolean {
|
||||
if (!name) return false;
|
||||
if (name.includes("/") || name.includes("\\") || name.includes("..")) return false;
|
||||
return /^[a-zA-Z0-9_.\-]+$/.test(name);
|
||||
}
|
||||
@@ -0,0 +1,76 @@
|
||||
/**
|
||||
* L2 Response Parser — extracts MMD generation results from LLM output.
|
||||
*/
|
||||
import { extractJson, extractMermaidFromFence } from "./json-utils.js";
|
||||
import type { L2ParsedResponse } from "../types.js";
|
||||
|
||||
interface RawL2Response {
|
||||
file_action?: string;
|
||||
mmd_content?: string | null;
|
||||
replace_blocks?: Array<{
|
||||
start_line?: number | string;
|
||||
end_line?: number | string;
|
||||
content?: string;
|
||||
}> | null;
|
||||
node_mapping?: Record<string, string>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse L2 LLM response into structured result.
|
||||
* Returns null if parsing fails completely.
|
||||
*/
|
||||
export function parseL2Response(raw: string): L2ParsedResponse | null {
|
||||
const parsed = extractJson<RawL2Response>(raw);
|
||||
if (!parsed || typeof parsed !== "object") {
|
||||
// Fallback: try extracting ```mermaid ... ``` code block
|
||||
const mmd = extractMermaidFromFence(raw);
|
||||
if (mmd) {
|
||||
return { fileAction: "write", mmdContent: mmd, nodeMapping: {} };
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
const fileAction = parsed.file_action === "replace" ? "replace" : "write";
|
||||
|
||||
// Extract mmd_content (may be wrapped in code fence)
|
||||
let mmdContent: string | undefined;
|
||||
if (fileAction === "write") {
|
||||
if (parsed.mmd_content) {
|
||||
mmdContent =
|
||||
extractMermaidFromFence(parsed.mmd_content) ?? parsed.mmd_content;
|
||||
} else {
|
||||
const fallbackMmd = extractMermaidFromFence(raw);
|
||||
if (fallbackMmd) mmdContent = fallbackMmd;
|
||||
}
|
||||
}
|
||||
|
||||
// Parse replace_blocks
|
||||
let replaceBlocks: L2ParsedResponse["replaceBlocks"] | undefined;
|
||||
if (fileAction === "replace" && Array.isArray(parsed.replace_blocks)) {
|
||||
replaceBlocks = [];
|
||||
for (const block of parsed.replace_blocks) {
|
||||
if (!block || typeof block !== "object") continue;
|
||||
const startLine = Number(block.start_line);
|
||||
const endLine = Number(block.end_line);
|
||||
if (isNaN(startLine) || isNaN(endLine)) continue;
|
||||
|
||||
let content = block.content ?? "";
|
||||
const extracted = extractMermaidFromFence(content);
|
||||
if (extracted) content = extracted;
|
||||
|
||||
replaceBlocks.push({ startLine, endLine, content });
|
||||
}
|
||||
}
|
||||
|
||||
// Parse node_mapping
|
||||
const nodeMapping: Record<string, string> = {};
|
||||
if (parsed.node_mapping && typeof parsed.node_mapping === "object") {
|
||||
for (const [key, value] of Object.entries(parsed.node_mapping)) {
|
||||
if (typeof value === "string") {
|
||||
nodeMapping[key] = value;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return { fileAction, mmdContent, replaceBlocks, nodeMapping };
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
/**
|
||||
* L1 Summarization Prompt — converts ToolPairs into OffloadEntry summaries.
|
||||
*/
|
||||
import type { ToolPair } from "../types.js";
|
||||
|
||||
const PARAMS_MAX_LEN = 500;
|
||||
const RESULT_MAX_LEN = 2000;
|
||||
const COMPRESS_THRESHOLD = 200;
|
||||
|
||||
export const L1_SYSTEM_PROMPT = `你是一个专为 AI 编码助手提供支持的"工具结果摘要器"。你的核心任务是深度理解当前的对话上下文,并将繁杂的工具调用与执行结果(一对toolcall和tool result整合成一条summary输出),提炼为高信息密度的 JSON 数组。
|
||||
|
||||
在生成摘要前,请务必进行以下内部思考:
|
||||
1. 任务对齐:结合最近的对话记录,识别用户当前的核心目标和最新意图。若上下文存在冲突,始终以最新的用户意图为准。
|
||||
2. 价值过滤:忽略工具如何工作的冗余细节,直接提取"发现了什么关键线索"、"做了什么关键动作"、"修改了什么具体内容"或"遇到了什么具体报错"。
|
||||
3. 影响评估:判断该结果对当前任务的实质性影响(例如:证实了某个假设、推进了哪一步、做出了什么决策,或因为什么报错导致了阻塞)。
|
||||
|
||||
【输出格式要求】
|
||||
你必须且只能输出一个合法的 JSON 对象数组 [{...}],每个对象**必须**包含以下字段:
|
||||
- "tool_call": 工具调用的简洁描述。处理规则如下:
|
||||
· 如果输入中该 tool pair 标记了 [NEEDS_COMPRESS],你必须将工具名+关键参数压缩为一句简洁的描述(≤150字符),保留工具名、操作目标(如文件路径、命令意图),省略内联脚本/大段内容的细节。
|
||||
示例:exec({"command":"python3 -c 'import csv; ...200行脚本...'"}) → "exec: 运行 Python (xx/xx/xx.sh,标明具体路径和文件)脚本分析 sales_channels.csv 数据质量"
|
||||
示例:write_file({"path":"/root/app.py","content":"...5000字符..."}) → "write_file: 写入 /root/app.py (Flask 应用主文件),大致内容是……"
|
||||
· 如果未标记 [NEEDS_COMPRESS],直接简述工具与参数即可(系统会用原始值覆盖)。
|
||||
- "summary": 融合上述思考的精炼总结(≤200个字符)。必须一针见血地说清楚结果的业务价值,以及它对任务的推进/阻塞作用。
|
||||
- "tool_call_id": 原始的 tool_call_id(必须原样透传)。
|
||||
- "timestamp": 原始的 ISO 8601 时间戳(必须原样透传)。
|
||||
- "score"(**必填**): 结合信息密度和任务目的分析summary对于原文的可替代性,范围在0-10之间,越接近10表示summary越能替代原文。
|
||||
|
||||
【严格规则】
|
||||
只允许输出纯 JSON 数组,严禁输出思考过程或其他解释性文本。`;
|
||||
|
||||
/**
|
||||
* Build the L1 user prompt for summarization.
|
||||
*/
|
||||
export function buildL1UserPrompt(
|
||||
recentContext: string,
|
||||
pairs: ToolPair[],
|
||||
): string {
|
||||
const parts: string[] = [];
|
||||
|
||||
parts.push("## 最近的对话上下文(用于理解当前任务):");
|
||||
parts.push(recentContext || "(无可用上下文)");
|
||||
parts.push("\n## Tool call/result pairs to summarize:");
|
||||
|
||||
for (let i = 0; i < pairs.length; i++) {
|
||||
const p = pairs[i];
|
||||
const paramsStr = truncate(stringify(p.params), PARAMS_MAX_LEN);
|
||||
const resultStr = truncate(stringify(p.result), RESULT_MAX_LEN);
|
||||
const canonical = `${p.toolName}(${stringify(p.params)})`;
|
||||
const needsCompress = canonical.length > COMPRESS_THRESHOLD;
|
||||
|
||||
parts.push(`--- Tool Pair ${i + 1} ---`);
|
||||
parts.push(`tool_call_id: ${p.toolCallId}`);
|
||||
parts.push(`timestamp: ${p.timestamp}`);
|
||||
if (needsCompress) {
|
||||
parts.push(`Tool: ${p.toolName} [NEEDS_COMPRESS]`);
|
||||
} else {
|
||||
parts.push(`Tool: ${p.toolName}`);
|
||||
}
|
||||
parts.push(`Params: ${paramsStr}`);
|
||||
parts.push(`Result: ${resultStr}\n`);
|
||||
}
|
||||
|
||||
parts.push("Summarize each pair into the JSON array format described.");
|
||||
return parts.join("\n");
|
||||
}
|
||||
|
||||
function stringify(value: unknown): string {
|
||||
if (value == null) return "";
|
||||
if (typeof value === "string") return value;
|
||||
try {
|
||||
return JSON.stringify(value);
|
||||
} catch {
|
||||
return String(value);
|
||||
}
|
||||
}
|
||||
|
||||
function truncate(s: string, maxLen: number): string {
|
||||
if (s.length <= maxLen) return s;
|
||||
return s.slice(0, maxLen) + "...";
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
/**
|
||||
* L1.5 Task Judgment Prompt — determines task lifecycle.
|
||||
*/
|
||||
import type { MmdMeta } from "../types.js";
|
||||
|
||||
export const L15_SYSTEM_PROMPT = `你是一个面向 AI 编码助手的"任务生命周期门神"。
|
||||
你的职责是交叉分析提供的三个输入源,精准研判任务状态,并输出纯 JSON 对象。
|
||||
|
||||
【输入数据利用指南(必须遵循的思考链路)】
|
||||
1. 第一步 - 剖析 recentMessages(识别意图):根据当前和历史对话,提取用户最新回复的核心诉求。判断是"继续排查"、"宣布完工(如:跑通了)"、"单轮闲聊问答"还是"开启全新需求"。
|
||||
2. 第二步 - 对齐 currentMmd(评估当前基线):将用户的最新意图与 currentMmd 的完整 Mermaid 内容进行比对——关注 taskGoal、各节点的 status(done/doing/todo)以及 summary。如果诉求完全超出了当前图表的范畴或目标已实现(所有节点 done 且无后续),则 taskCompleted 为 true。若仍在解决图表中的子问题(包括 doing 节点或修 bug),则为 false。(如果没有currentMmd,就只根据当前对话和历史对话来判断是否继续任务)
|
||||
3. 第三步 - 检索 availableMmds(判断是否延续):如果判定要开启新任务(isLongTask=true 且 taskCompleted=true/当前无任务),必须扫描 availableMmds 的 taskGoal 和时间信息。若新诉求与列表中某个旧任务高度重合(如回到昨天没做完的模块),则是延续(isContinuation=true)。
|
||||
|
||||
【严格 JSON 输出格式】
|
||||
务必输出合法的纯 JSON 对象,格式如下:
|
||||
{
|
||||
"taskCompleted": boolean, // 当前任务是否已结束(如果 currentMmd 为 none,这里必须填 true)
|
||||
"isLongTask": boolean, // 最新诉求是否是需要多步操作的复杂工程(普通技术问答、闲聊填 false)
|
||||
"isContinuation": boolean, // 是否在延续 availableMmds 中的历史任务
|
||||
"continuationMmdFile": "string|null", // 若延续旧任务,精确填入 availableMmds 中的文件名(不含路径前缀),否则为 null
|
||||
"newTaskLabel": "string|null" // 若是全新长任务,生成简短标签(≤30字符,kebab-case,如 "refactor-api"),否则为 null
|
||||
}
|
||||
|
||||
只输出纯 JSON 对象,绝不允许包含解释文字。`;
|
||||
|
||||
export interface L15CurrentMmd {
|
||||
filename: string;
|
||||
content: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the L1.5 user prompt for task judgment.
|
||||
*/
|
||||
export function buildL15UserPrompt(
|
||||
recentMessages: string,
|
||||
currentMmd: L15CurrentMmd | null,
|
||||
metas: MmdMeta[],
|
||||
): string {
|
||||
const parts: string[] = [];
|
||||
|
||||
parts.push("## 1. 最近的对话上下文 (Recent messages):");
|
||||
parts.push(recentMessages);
|
||||
parts.push("\n## 2. 当前挂载的任务图 (Active Mermaid — 完整内容):");
|
||||
|
||||
if (currentMmd && currentMmd.filename) {
|
||||
parts.push(`**File:** ${currentMmd.filename}`);
|
||||
parts.push(`\n\`\`\`mermaid\n${currentMmd.content}\n\`\`\``);
|
||||
} else {
|
||||
parts.push("(none - 当前处于闲置状态,无活跃任务)");
|
||||
}
|
||||
|
||||
parts.push("\n## 3. 历史可用的任务图 (Available Mermaid task files):");
|
||||
|
||||
if (metas.length === 0) {
|
||||
parts.push("(none - 暂无历史长任务)");
|
||||
} else {
|
||||
for (const m of metas) {
|
||||
const total = m.doneCount + m.doingCount + m.todoCount;
|
||||
parts.push(`- **${m.filename}**`);
|
||||
parts.push(` taskGoal: ${m.taskGoal}`);
|
||||
parts.push(
|
||||
` progress: ${m.doneCount}/${total} done, ${m.doingCount} doing, ${m.todoCount} todo`,
|
||||
);
|
||||
if (m.updatedTime) {
|
||||
parts.push(` lastUpdated: ${m.updatedTime}`);
|
||||
}
|
||||
if (m.nodeSummaries && m.nodeSummaries.length > 0) {
|
||||
parts.push(" recentNodes:");
|
||||
for (const n of m.nodeSummaries) {
|
||||
parts.push(` - [${n.nodeId}] (${n.status}) ${n.summary}`);
|
||||
}
|
||||
}
|
||||
parts.push("");
|
||||
}
|
||||
}
|
||||
|
||||
parts.push(
|
||||
"请严格根据系统指令的【三步思考链路】进行研判,并输出合法的 JSON 对象。",
|
||||
);
|
||||
return parts.join("\n");
|
||||
}
|
||||
@@ -0,0 +1,113 @@
|
||||
/**
|
||||
* L2 MMD Generation Prompt — generates/updates Mermaid flowcharts.
|
||||
*/
|
||||
import type { OffloadEntry } from "../types.js";
|
||||
|
||||
export const L2_SYSTEM_PROMPT = `你是一个究极实用主义的 AI 任务拓扑架构师与视觉叙事者。
|
||||
你的核心逻辑是用尽量少的字符表达尽量多的信息,让LLM模型能看懂,不是为人类服务,尽量减少无用的视觉符号。任务是将底层工具调用记录,升维映射为一张高度语义化、表现力丰富且极度克制的 Mermaid (flowchart TD) 认知状态机。你要根据当前任务和意图,归纳"过去",要思考"未来"如何用这些已有的信息(你只需要记录已有信息,不需要写下一步规划)并标记"雷区"。保持图表的高度概括性。
|
||||
|
||||
【高阶认知与拓扑指南(你的自主权与极简原则)】
|
||||
1. 弹性聚合:你拥有决定节点拆合的完全自主权。对于连续的、意图相同的常规动作(如连续查看多个文件以了解上下文),建议合并为一个宏观节点;,但保留关键转折点或重大发现为独立节点。图表必须保持宏观和克制,绝不事无巨细地记流水账。
|
||||
2. 认知墓碑 (防重蹈覆辙):遇到彻底走不通的死胡同或引发严重报错的废弃方案,可以建立警示节点(status: blocked)(如果是价值不高的fail信息则不需要记录)。
|
||||
3. 结论导向的摘要:节点的 summary(注意:尽量小于150字)应聚焦于"得出了什么结论"或"发生了什么实质改变",而非罗列琐碎的数据或参数,记得保持极简原则。
|
||||
4. 要实事求是,你的任务是记录并归纳已经发生的事情,不是规划未来的具体操作,未发生的节点不要写,记录的已发生节点要有对应的消息来源(对应标注node_id)。
|
||||
【符号即语义:高维认知字典(你的核心武器)】为了极致压缩 Token 并为你下一步推理提供"认知锚点",请自由使用不同的mmd形状来代表不同的节点逻辑。让形状替你说话,省略冗余的文字描述。
|
||||
|
||||
【高度自由的拓扑与极简法则】
|
||||
1. 语义浓缩:既然形状已经表达了"领域",你的 summary 必须极其精简(≤150字),如"发现死锁"、"依赖冲突"、"已修复"。
|
||||
2. 弹性拓扑:自主使用带标签的连线(-->|测试失败|)和虚线(-.->|参考|)来构建"依赖树"和"假设验证环"。不要记流水账。
|
||||
3. 动态更新 (Token 极简):
|
||||
- replace (增量微调):仅修改现有节点的状态、时间戳、短文本或追加极少节点时。
|
||||
- write (全量重写):逻辑大洗牌、重构图表或初始化时。
|
||||
注意:Existing Mermaid content 中每行开头都带有行号标记(如 "L1: ..."),这些行号仅供你在 replace 模式中引用,不是 MMD 内容的一部分。
|
||||
|
||||
【严格的工程底线】
|
||||
1.节点标准格式:NodeID["阶段名: 宏观动作简述<br/>status: done|doing|paused|blocked <br/>summary: 核心结论摘要<br/>Timestamp: ISO8601"]
|
||||
2. 全员归宿映射:输入的每一个新 tool_call_id,都必须在 node_mapping 中被分配到一个 Node ID;MMD里的每一个node都应该有源头的tool_call消息来源,不能乱编,绝对不允许遗漏!(Node_id和tool_call_id是一对多的关系)
|
||||
3. 你可以通过各种整合方法,尽量把更新后mmd文件大小控制在4000字以内
|
||||
|
||||
【严格时间戳与元数据规则】
|
||||
1. 顶部元数据(必填):%%{ "taskGoal": "一句话总结此次任务的目标(可动态更新)", "progress(0-100)": "进度百分比(严格点,几乎确认完成再打到90+)", createdTime": "ISO时间", "updatedTime": "ISO时间" }%%(updatedTime为node中的最新时间)。
|
||||
2. 节点内时间:如果合并了多个新条目,节点内的 Timestamp 必须取其中最新的 ISO 时间。
|
||||
|
||||
【严格 JSON 输出格式】
|
||||
务必正确转义双引号。所有 Mermaid 代码(无论是 mmd_content 还是 replace_blocks 中的 content)都必须用 \`\`\`mermaid ... \`\`\` 代码块包裹起来。必须输出如下 JSON 结构:
|
||||
{
|
||||
"file_action": "replace 或 write",
|
||||
"mmd_content": "完整的、带转义的 .mmd 代码,必须用 \`\`\`mermaid ... \`\`\` 包裹。(仅在 file_action 为 write 时填写,否则必须设为 null)",
|
||||
"replace_blocks": [
|
||||
{
|
||||
"start_line": "需要更新范围的起始行号(整数,对应 Existing Mermaid content 中的 L 标号)",
|
||||
"end_line": "需要更新范围的结束行号(整数,包含该行)。要在某行之前插入新内容而不删除任何行,将 start_line 设为该行号,end_line 设为 start_line - 1",
|
||||
"content": "替换后的新内容(不需要带行号前缀),必须用 \`\`\`mermaid ... \`\`\` 包裹"
|
||||
}
|
||||
],
|
||||
"node_mapping": {
|
||||
"tool_call_id_1": "001-N1",
|
||||
"tool_call_id_2": "001-N1"
|
||||
}
|
||||
}
|
||||
|
||||
注意:node_mapping 中的 Node ID 必须是 MMD 中实际使用的完整 ID(包含 MMD prefix,如 "001-N1"),不能只写短 ID(如 "N1")。
|
||||
仅输出纯 JSON 对象,绝不允许包含任何解释。`;
|
||||
|
||||
/**
|
||||
* Build the L2 user prompt for MMD generation.
|
||||
*/
|
||||
export function buildL2UserPrompt(opts: {
|
||||
existingMmd: string | null;
|
||||
entries: OffloadEntry[];
|
||||
recentHistory?: string | null;
|
||||
currentTurn?: string | null;
|
||||
taskLabel: string;
|
||||
mmdPrefix: string;
|
||||
charCount: number;
|
||||
}): string {
|
||||
const { existingMmd, entries, recentHistory, currentTurn, taskLabel, mmdPrefix, charCount } = opts;
|
||||
const parts: string[] = [];
|
||||
|
||||
// History section
|
||||
if (recentHistory) {
|
||||
parts.push(`## 近期对话历史:\n${recentHistory}`);
|
||||
} else {
|
||||
parts.push("## 近期对话历史:\n(无可用历史)");
|
||||
}
|
||||
|
||||
if (currentTurn) {
|
||||
parts.push(`\n## 当前最新一轮:\n${currentTurn}`);
|
||||
}
|
||||
|
||||
parts.push(`\n## MMD prefix: ${mmdPrefix}`);
|
||||
parts.push(`(所有节点 ID 必须以此前缀开头,如 ${mmdPrefix}-N1, ${mmdPrefix}-N2...)`);
|
||||
parts.push(`\n## Current task label: ${taskLabel}`);
|
||||
|
||||
// Char count warning
|
||||
if (charCount > 2500) {
|
||||
parts.push(`\n## Current MMD size: ${charCount} chars (budget: 4000 chars)`);
|
||||
parts.push("⚠ 接近上限,请积极合并节点、精简 summary,优先使用 replace 模式微调而非 write 全量重写。");
|
||||
} else if (charCount > 2000) {
|
||||
parts.push(`\n## Current MMD size: ${charCount} chars (budget: 4000 chars)`);
|
||||
parts.push("注意控制增长,合并同类节点。");
|
||||
}
|
||||
|
||||
// Existing MMD with line numbers
|
||||
parts.push("\n## Existing Mermaid content:");
|
||||
if (existingMmd) {
|
||||
const lines = existingMmd.split("\n");
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
parts.push(`L${i + 1}: ${lines[i]}`);
|
||||
}
|
||||
} else {
|
||||
parts.push("(empty — create new)");
|
||||
}
|
||||
|
||||
// New entries
|
||||
parts.push("\n## New offload entries to incorporate:");
|
||||
for (let i = 0; i < entries.length; i++) {
|
||||
const e = entries[i];
|
||||
parts.push(`${i + 1}. [${e.tool_call_id}] ${e.tool_call} → ${e.summary} (${e.timestamp})`);
|
||||
}
|
||||
|
||||
parts.push("\n请根据系统指令生成/更新 Mermaid 流程图,并输出合法的 JSON 对象(含 node_mapping)。");
|
||||
return parts.join("\n");
|
||||
}
|
||||
@@ -0,0 +1,88 @@
|
||||
/**
|
||||
* Offload V2 Router — route registration and dispatch.
|
||||
*/
|
||||
import type http from "node:http";
|
||||
import type { StorageAdapter } from "../core/storage/adapter.js";
|
||||
import type { IStateBackend } from "../core/state/types.js";
|
||||
import type { OffloadExecutorConfig } from "./types.js";
|
||||
import { defaultOffloadConfig } from "./types.js";
|
||||
import { parseV2Auth, successEnvelope, errorEnvelope, makeRequestId } from "../gateway/v2-router.js";
|
||||
import { handleIngest } from "./ingest-handler.js";
|
||||
import { handleMmdQuery } from "./mmd-handler.js";
|
||||
import { handleCompaction } from "./compact/compaction-handler.js";
|
||||
import { MmdQuerySchema } from "./schemas.js";
|
||||
|
||||
export interface OffloadV2Deps {
|
||||
resolveStorage?: (instanceId: string) => Promise<StorageAdapter | undefined>;
|
||||
getStorage: () => StorageAdapter | undefined;
|
||||
logger: { info: (...args: unknown[]) => void; warn: (...args: unknown[]) => void; error: (...args: unknown[]) => void };
|
||||
stateBackend?: IStateBackend;
|
||||
config?: OffloadExecutorConfig;
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle offload V2 routes. Returns true if the request was handled.
|
||||
*/
|
||||
export async function handleOffloadV2Route(
|
||||
req: http.IncomingMessage,
|
||||
res: http.ServerResponse,
|
||||
pathname: string,
|
||||
method: string,
|
||||
parseJsonBody: <T>(req: http.IncomingMessage) => Promise<T>,
|
||||
sendJson: (res: http.ServerResponse, status: number, body: unknown) => void,
|
||||
deps: OffloadV2Deps,
|
||||
): Promise<boolean> {
|
||||
if (!pathname.startsWith("/v2/offload/")) return false;
|
||||
|
||||
const requestId = makeRequestId();
|
||||
|
||||
// Auth
|
||||
const auth = parseV2Auth(req, res, requestId, sendJson);
|
||||
if (!auth) return true; // 401 already sent
|
||||
|
||||
// Resolve storage
|
||||
const storage =
|
||||
(await deps.resolveStorage?.(auth.serviceId)) ?? deps.getStorage();
|
||||
if (!storage) {
|
||||
sendJson(res, 503, errorEnvelope(503, "Storage unavailable", requestId));
|
||||
return true;
|
||||
}
|
||||
|
||||
const config = deps.config ?? defaultOffloadConfig();
|
||||
// Normalize trailing slash for consistent route matching
|
||||
const normalizedPath = pathname.endsWith("/") && pathname.length > 1 ? pathname.slice(0, -1) : pathname;
|
||||
const route = `${method} ${normalizedPath}`;
|
||||
|
||||
switch (route) {
|
||||
case "POST /v2/offload/ingest":
|
||||
await handleIngest(req, res, auth, {
|
||||
storage,
|
||||
stateBackend: deps.stateBackend,
|
||||
config,
|
||||
logger: deps.logger,
|
||||
}, requestId, parseJsonBody, sendJson, successEnvelope, errorEnvelope);
|
||||
return true;
|
||||
|
||||
case "POST /v2/offload/query-mmd": {
|
||||
const body = await parseJsonBody<{ session_id?: string; limit?: number }>(req);
|
||||
const parsed = MmdQuerySchema.safeParse(body);
|
||||
if (!parsed.success) {
|
||||
sendJson(res, 400, errorEnvelope(400, "missing or invalid session_id in body", requestId));
|
||||
return true;
|
||||
}
|
||||
await handleMmdQuery(req, res, auth, storage, requestId, sendJson, successEnvelope, errorEnvelope, parsed.data.session_id, parsed.data.limit);
|
||||
return true;
|
||||
}
|
||||
|
||||
case "POST /v2/offload/compact":
|
||||
await handleCompaction(req, res, auth, {
|
||||
storage,
|
||||
config,
|
||||
logger: deps.logger,
|
||||
}, requestId, parseJsonBody, sendJson, successEnvelope, errorEnvelope);
|
||||
return true;
|
||||
|
||||
default:
|
||||
return false;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,80 @@
|
||||
/**
|
||||
* Offload Server — Request validation schemas (Zod).
|
||||
*/
|
||||
import { z } from "zod";
|
||||
|
||||
/** Safe session ID: alphanumeric, underscore, hyphen, dot, colon allowed. No slashes or path traversal. Max 500 chars. */
|
||||
const safeSessionId = z.string().min(1).max(500, {
|
||||
message: "sessionId must not exceed 500 characters",
|
||||
}).regex(/^[a-zA-Z0-9_.\-:]+$/, {
|
||||
message: "Must only contain alphanumeric, underscore, hyphen, dot, or colon characters",
|
||||
});
|
||||
|
||||
const ToolPairSchema = z.object({
|
||||
tool_name: z.string(),
|
||||
tool_call_id: z.string(),
|
||||
params: z.unknown(),
|
||||
result: z.unknown(),
|
||||
error: z.string().optional(),
|
||||
timestamp: z.string(),
|
||||
duration_ms: z.number().optional(),
|
||||
});
|
||||
|
||||
/** Recent message item: user/assistant text only (no tool_call/tool_result). */
|
||||
const RecentMessageSchema = z.object({
|
||||
role: z.enum(["user", "assistant"]),
|
||||
content: z.string(),
|
||||
});
|
||||
|
||||
export const IngestRequestSchema = z
|
||||
.object({
|
||||
session_id: safeSessionId,
|
||||
tool_pairs: z.array(ToolPairSchema).default([]),
|
||||
/** Current user prompt that triggers L1.5 task judgment. Must be non-empty (whitespace-only is rejected). */
|
||||
prompt: z.string().trim().min(1, { message: "prompt must not be empty or whitespace-only" }).optional(),
|
||||
/** Recent history messages (user/assistant only, no tool calls). */
|
||||
recent_messages: z.array(RecentMessageSchema).optional(),
|
||||
})
|
||||
.refine(
|
||||
(data) => data.tool_pairs.length > 0 || (data.prompt && data.prompt.length > 0),
|
||||
{ message: "Either tool_pairs must be non-empty or prompt must be provided" },
|
||||
);
|
||||
|
||||
export type IngestRequest = z.infer<typeof IngestRequestSchema>;
|
||||
|
||||
/**
|
||||
* Each message must have non-empty `role` and `content` fields.
|
||||
* Intentionally lenient: any non-empty string role is accepted to support
|
||||
* OpenAI / Anthropic / OpenClaw-wrapped formats without over-specifying.
|
||||
*/
|
||||
const CompactionMessageSchema = z
|
||||
.record(z.string(), z.unknown())
|
||||
.refine(
|
||||
(msg) => typeof msg["role"] === "string" && (msg["role"] as string).length > 0,
|
||||
{ message: "Each message must have a non-empty 'role' field" },
|
||||
);
|
||||
|
||||
export const CompactionRequestSchema = z.object({
|
||||
session_id: safeSessionId,
|
||||
messages: z.array(CompactionMessageSchema),
|
||||
ratio: z.number().min(0).max(2),
|
||||
});
|
||||
|
||||
export type CompactionRequest = z.infer<typeof CompactionRequestSchema>;
|
||||
|
||||
/** Extended compaction schema with token metadata for L3 compression. */
|
||||
export const CompactionRequestSchemaV2 = z.object({
|
||||
session_id: safeSessionId,
|
||||
messages: z.array(CompactionMessageSchema),
|
||||
ratio: z.number().min(0).max(2),
|
||||
context_window: z.number().int().min(1),
|
||||
total_tokens: z.number().int().min(0),
|
||||
message_tokens: z.array(z.number()).optional(),
|
||||
});
|
||||
|
||||
export type CompactionRequestV2 = z.infer<typeof CompactionRequestSchemaV2>;
|
||||
|
||||
export const MmdQuerySchema = z.object({
|
||||
session_id: safeSessionId,
|
||||
limit: z.number().int().min(1).optional(),
|
||||
});
|
||||
@@ -0,0 +1,29 @@
|
||||
/**
|
||||
* Offload Session Utilities — unified sessionId sanitization.
|
||||
*
|
||||
* sessionId: opaque identifier from client (free format, e.g. "main:main", "coding:session-001")
|
||||
* sanitizedSessionId: filesystem-safe version used for:
|
||||
* - File/directory paths (offload/{sanitizedSessionId}/)
|
||||
* - Distributed lock keys
|
||||
* - Task queue session identifiers
|
||||
*
|
||||
* Conversion: replace all non-alphanumeric/dot/hyphen/underscore chars with "_"
|
||||
*/
|
||||
|
||||
/**
|
||||
* Convert a raw sessionId to a filesystem-safe string.
|
||||
* Replaces colons and other unsafe characters with underscores.
|
||||
*/
|
||||
export function sanitizeSessionId(sessionId: string): string {
|
||||
return sessionId.replace(/[^a-zA-Z0-9._\-]/g, "_");
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the offload storage base path for a session.
|
||||
*/
|
||||
export function buildOffloadBasePath(sessionId: string): string {
|
||||
return `offload/${sanitizeSessionId(sessionId)}`;
|
||||
}
|
||||
|
||||
// Legacy compat: keep old name as alias (will remove after all callers migrated)
|
||||
export const toSessionId = sanitizeSessionId;
|
||||
@@ -0,0 +1,132 @@
|
||||
/**
|
||||
* Task Transition logic — manages MMD file creation/switching based on L1.5 judgment.
|
||||
*/
|
||||
import type { StorageAdapter } from "../core/storage/adapter.js";
|
||||
import type { OffloadState, TaskJudgment, MmdMeta } from "./types.js";
|
||||
|
||||
/**
|
||||
* Apply task transition based on L1.5 judgment.
|
||||
* Mutates state in place; may create new MMD file in COS.
|
||||
* @param basePath - Full storage base path, e.g. "offload/default/agent_main_tui-xxx"
|
||||
*/
|
||||
export async function handleTaskTransition(
|
||||
state: OffloadState,
|
||||
judgment: TaskJudgment,
|
||||
storage: StorageAdapter,
|
||||
basePath: string,
|
||||
): Promise<void> {
|
||||
if (judgment.taskCompleted && judgment.isLongTask && !judgment.isContinuation && judgment.newTaskLabel) {
|
||||
// CASE 1: New long task → create new MMD file
|
||||
const filename = await generateMmdFilename(storage, basePath, judgment.newTaskLabel);
|
||||
await storage.writeFile(`${basePath}/mmds/${filename}`, "");
|
||||
state.activeMmdFile = filename;
|
||||
} else if (judgment.taskCompleted && judgment.isContinuation && judgment.continuationMmdFile) {
|
||||
// CASE 2: Continue historical task → switch to that MMD
|
||||
state.activeMmdFile = judgment.continuationMmdFile;
|
||||
} else if (judgment.taskCompleted && !judgment.isLongTask) {
|
||||
// CASE 3: Short task / casual chat → clear active MMD
|
||||
state.activeMmdFile = null;
|
||||
} else if (!judgment.taskCompleted && judgment.isLongTask && !state.activeMmdFile) {
|
||||
// CASE 5: Task in progress but no active MMD yet → create one
|
||||
const label = judgment.newTaskLabel || "current-task";
|
||||
const filename = await generateMmdFilename(storage, basePath, label);
|
||||
await storage.writeFile(`${basePath}/mmds/${filename}`, "");
|
||||
state.activeMmdFile = filename;
|
||||
}
|
||||
// CASE 4: !taskCompleted + activeMmdFile already set → keep unchanged
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate a new MMD filename with auto-incrementing sequence number.
|
||||
* Format: "003-refactor-api.mmd"
|
||||
*/
|
||||
async function generateMmdFilename(
|
||||
storage: StorageAdapter,
|
||||
basePath: string,
|
||||
label: string,
|
||||
): Promise<string> {
|
||||
const mmdsPrefix = `${basePath}/mmds/`;
|
||||
const existingFiles = await storage.readdirNames(mmdsPrefix, ".mmd");
|
||||
|
||||
let maxSeq = 0;
|
||||
for (const f of existingFiles) {
|
||||
const match = f.match(/^(\d+)-/);
|
||||
if (match) {
|
||||
const seq = parseInt(match[1], 10);
|
||||
if (seq > maxSeq) maxSeq = seq;
|
||||
}
|
||||
}
|
||||
|
||||
const nextSeq = String(maxSeq + 1).padStart(3, "0");
|
||||
const safeLabel = label
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9-]/g, "-")
|
||||
.replace(/-+/g, "-")
|
||||
.replace(/^-|-$/g, "")
|
||||
.slice(0, 30);
|
||||
|
||||
return `${nextSeq}-${safeLabel || "task"}.mmd`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract MmdMeta from MMD file content (parses %%{ ... }%% header).
|
||||
*/
|
||||
export function extractMmdMeta(filename: string, content: string): MmdMeta {
|
||||
const defaults: MmdMeta = {
|
||||
filename,
|
||||
taskGoal: "",
|
||||
doneCount: 0,
|
||||
doingCount: 0,
|
||||
todoCount: 0,
|
||||
updatedTime: null,
|
||||
nodeSummaries: [],
|
||||
};
|
||||
|
||||
if (!content) return defaults;
|
||||
|
||||
// Parse %%{ ... }%% metadata line
|
||||
const metaMatch = content.match(/%%\{([\s\S]*?)\}%%/);
|
||||
if (metaMatch) {
|
||||
try {
|
||||
const meta = JSON.parse(`{${metaMatch[1]}}`);
|
||||
defaults.taskGoal = meta.taskGoal ?? "";
|
||||
defaults.updatedTime = meta.updatedTime ?? null;
|
||||
} catch {
|
||||
// ignore parse errors
|
||||
}
|
||||
}
|
||||
|
||||
// Count node statuses and extract summaries
|
||||
const nodeRegex = /(\w[\w-]*)\["([^"]*?)"\]/g;
|
||||
let nodeMatch;
|
||||
while ((nodeMatch = nodeRegex.exec(content)) !== null) {
|
||||
const nodeId = nodeMatch[1];
|
||||
const nodeContent = nodeMatch[2];
|
||||
const statusM = nodeContent.match(/status:\s*(done|doing|todo|paused|blocked)/i);
|
||||
const summaryM = nodeContent.match(/summary:\s*(.+?)(?:<br\/>|$)/i);
|
||||
if (statusM) {
|
||||
const s = statusM[1].toLowerCase();
|
||||
if (s === "done") defaults.doneCount++;
|
||||
else if (s === "doing") defaults.doingCount++;
|
||||
else if (s === "todo") defaults.todoCount++;
|
||||
defaults.nodeSummaries!.push({
|
||||
nodeId,
|
||||
status: s,
|
||||
summary: summaryM?.[1]?.trim() ?? "",
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Fallback: if regex didn't catch statuses, try simple pattern
|
||||
if (defaults.doneCount === 0 && defaults.doingCount === 0 && defaults.todoCount === 0) {
|
||||
const statusMatches = content.matchAll(/status:\s*(done|doing|todo|paused|blocked)/gi);
|
||||
for (const m of statusMatches) {
|
||||
const s = m[1].toLowerCase();
|
||||
if (s === "done") defaults.doneCount++;
|
||||
else if (s === "doing") defaults.doingCount++;
|
||||
else if (s === "todo") defaults.todoCount++;
|
||||
}
|
||||
}
|
||||
|
||||
return defaults;
|
||||
}
|
||||
@@ -0,0 +1,163 @@
|
||||
/**
|
||||
* Offload Server — Independent type definitions.
|
||||
* This module does NOT import from src/offload/.
|
||||
*/
|
||||
|
||||
// ─── ToolPair (ingest 写入 pending.jsonl) ────────────────────────────────────
|
||||
|
||||
export interface ToolPair {
|
||||
toolName: string;
|
||||
toolCallId: string;
|
||||
params: unknown;
|
||||
result: unknown;
|
||||
error?: string;
|
||||
timestamp: string;
|
||||
durationMs?: number;
|
||||
}
|
||||
|
||||
// ─── OffloadEntry (L1 产出, entries.jsonl 每行) ──────────────────────────────
|
||||
|
||||
export interface OffloadEntry {
|
||||
tool_call_id: string;
|
||||
tool_call: string;
|
||||
summary: string;
|
||||
timestamp: string;
|
||||
score: number;
|
||||
node_id: string | null;
|
||||
/** Full relative path to ref file storing original tool result (e.g. "offload/{sessionId}/refs/call_214.md"), readable via tdai_read_cos */
|
||||
result_ref?: string;
|
||||
}
|
||||
|
||||
// ─── TaskJudgment (L1.5 产出) ────────────────────────────────────────────────
|
||||
|
||||
export interface TaskJudgment {
|
||||
taskCompleted: boolean;
|
||||
isLongTask: boolean;
|
||||
isContinuation: boolean;
|
||||
continuationMmdFile?: string;
|
||||
newTaskLabel?: string;
|
||||
}
|
||||
|
||||
// ─── TaskBoundary (L1.5 写入 state.boundaries) ──────────────────────────────
|
||||
|
||||
export interface TaskBoundary {
|
||||
targetMmd: string | null;
|
||||
timestamp: string;
|
||||
}
|
||||
|
||||
// ─── MmdMeta (从 MMD 文件首行 %%{...}%% 解析) ──────────────────────────────
|
||||
|
||||
export interface MmdMeta {
|
||||
filename: string;
|
||||
taskGoal: string;
|
||||
doneCount: number;
|
||||
doingCount: number;
|
||||
todoCount: number;
|
||||
updatedTime?: string | null;
|
||||
nodeSummaries?: Array<{ nodeId: string; status: string; summary: string }>;
|
||||
}
|
||||
|
||||
// ─── OffloadState (state.json) ───────────────────────────────────────────────
|
||||
|
||||
export interface OffloadState {
|
||||
activeMmdFile: string | null;
|
||||
boundaries: TaskBoundary[];
|
||||
lastL15CreatedAt: number;
|
||||
}
|
||||
|
||||
// ─── CompactState (compact-state.json) ───────────────────────────────────────
|
||||
|
||||
export interface CompactState {
|
||||
confirmedOffloadIds: string[];
|
||||
deletedOffloadIds: string[];
|
||||
lastCompactedAt: string;
|
||||
}
|
||||
|
||||
// ─── L2 Parsed Response ──────────────────────────────────────────────────────
|
||||
|
||||
export interface L2ParsedResponse {
|
||||
fileAction: "write" | "replace";
|
||||
mmdContent?: string;
|
||||
replaceBlocks?: Array<{
|
||||
startLine: number;
|
||||
endLine: number;
|
||||
content: string;
|
||||
}>;
|
||||
nodeMapping: Record<string, string>;
|
||||
}
|
||||
|
||||
// ─── Configuration ───────────────────────────────────────────────────────────
|
||||
|
||||
export interface OffloadExecutorConfig {
|
||||
forceTriggerThreshold: number;
|
||||
pendingMaxAgeSeconds: number;
|
||||
|
||||
l1Model: string;
|
||||
l1Temperature: number;
|
||||
l1MaxTokens: number;
|
||||
l1TimeoutMs: number;
|
||||
|
||||
l15Model: string;
|
||||
l15Temperature: number;
|
||||
l15MaxTokens: number;
|
||||
l15TimeoutMs: number;
|
||||
|
||||
l2Model: string;
|
||||
l2Temperature: number;
|
||||
l2MaxTokens: number;
|
||||
l2TimeoutMs: number;
|
||||
l2NullThreshold: number;
|
||||
|
||||
mildOffloadRatio: number;
|
||||
aggressiveCompressRatio: number;
|
||||
emergencyCompressRatio: number;
|
||||
|
||||
maxRetries: number;
|
||||
}
|
||||
|
||||
export function defaultOffloadConfig(): OffloadExecutorConfig {
|
||||
return {
|
||||
forceTriggerThreshold: 4,
|
||||
pendingMaxAgeSeconds: 30,
|
||||
|
||||
l1Model: "", // uses gateway llm.model
|
||||
l1Temperature: 0.3,
|
||||
l1MaxTokens: 8000,
|
||||
l1TimeoutMs: 120_000,
|
||||
|
||||
l15Model: "", // uses gateway llm.model
|
||||
l15Temperature: 0.2,
|
||||
l15MaxTokens: 3000,
|
||||
l15TimeoutMs: 120_000,
|
||||
|
||||
l2Model: "", // uses gateway llm.model
|
||||
l2Temperature: 0.4,
|
||||
l2MaxTokens: 16000,
|
||||
l2TimeoutMs: 120_000,
|
||||
l2NullThreshold: 6,
|
||||
|
||||
mildOffloadRatio: 0.5,
|
||||
aggressiveCompressRatio: 0.85,
|
||||
emergencyCompressRatio: 0.95,
|
||||
|
||||
maxRetries: 3,
|
||||
};
|
||||
}
|
||||
|
||||
// ─── Default empty state ─────────────────────────────────────────────────────
|
||||
|
||||
export function defaultOffloadState(): OffloadState {
|
||||
return {
|
||||
activeMmdFile: null,
|
||||
boundaries: [],
|
||||
lastL15CreatedAt: 0,
|
||||
};
|
||||
}
|
||||
|
||||
export function defaultCompactState(): CompactState {
|
||||
return {
|
||||
confirmedOffloadIds: [],
|
||||
deletedOffloadIds: [],
|
||||
lastCompactedAt: "",
|
||||
};
|
||||
}
|
||||
@@ -42,6 +42,9 @@ export interface TaskExecutor {
|
||||
executeL2(task: TaskPayload, signal?: AbortSignal): Promise<void>;
|
||||
executeL3(task: TaskPayload, signal?: AbortSignal): Promise<void>;
|
||||
executeFlush?(task: TaskPayload, signal?: AbortSignal): Promise<void>;
|
||||
executeOffloadL1?(task: TaskPayload, signal?: AbortSignal): Promise<void>;
|
||||
executeOffloadL15?(task: TaskPayload, signal?: AbortSignal): Promise<void>;
|
||||
executeOffloadL2?(task: TaskPayload, signal?: AbortSignal): Promise<void>;
|
||||
}
|
||||
|
||||
export interface PipelineWorkerConfig {
|
||||
@@ -249,11 +252,67 @@ export class PipelineWorker {
|
||||
const lockKey = this.getLockKey(task);
|
||||
const retryCount = (task.data?.retryCount as number) ?? 0;
|
||||
|
||||
// Lock-free path: offload-l1 doesn't need distributed lock
|
||||
if (lockKey === null) {
|
||||
this.runningTasks.set(task.id, task);
|
||||
try {
|
||||
await this.executeTask(task, undefined);
|
||||
|
||||
// ACK
|
||||
const msgId = (task as any)._msgId;
|
||||
if (msgId) await this.backend.ackTask(msgId);
|
||||
|
||||
this.metrics.tasksCompleted++;
|
||||
this.logger?.debug?.(`${TAG} Task completed (lock-free): ${task.type} [${task.instanceId}/${task.sessionId}]`);
|
||||
} catch (err) {
|
||||
const errMsg = err instanceof Error ? err.message : String(err);
|
||||
this.metrics.tasksFailed++;
|
||||
|
||||
if (retryCount < this.config.maxRetries) {
|
||||
const delay = this.config.retryBaseDelayMs * Math.pow(3, retryCount);
|
||||
this.logger.warn(`${TAG} Task failed (lock-free, retry ${retryCount + 1}/${this.config.maxRetries}, delay=${delay}ms): ${errMsg}`);
|
||||
const msgId = (task as any)._msgId;
|
||||
if (msgId) { try { await this.backend.ackTask(msgId); } catch { /* best effort */ } }
|
||||
await this.sleep(delay);
|
||||
await this.reEnqueue(task, retryCount + 1);
|
||||
this.metrics.tasksRetried++;
|
||||
} else {
|
||||
await this.moveToDeadLetter(task, errMsg, retryCount);
|
||||
}
|
||||
} finally {
|
||||
this.runningTasks.delete(task.id);
|
||||
|
||||
// Deferred enqueue (same as locked path)
|
||||
const deferred = (task as any)._deferredEnqueue as TaskPayload[] | undefined;
|
||||
if (deferred?.length) {
|
||||
for (const dTask of deferred) {
|
||||
try {
|
||||
await this.backend.enqueueTask(dTask);
|
||||
this.logger?.debug?.(`${TAG} Deferred enqueue: ${dTask.type} [${dTask.id}]`);
|
||||
} catch (err) {
|
||||
this.logger?.warn?.(`${TAG} Deferred enqueue failed: ${err instanceof Error ? err.message : String(err)}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
// Step 1: 抢分布式锁
|
||||
const locked = await this.backend.acquireLock(lockKey, this.config.workerId, this.config.lockTtlMs);
|
||||
if (!locked) {
|
||||
this.metrics.lockConflicts++;
|
||||
|
||||
// offload-l2: skip immediately on lock conflict (idempotent timer will re-trigger)
|
||||
if (task.type === "offload-l2") {
|
||||
this.logger?.debug?.(`${TAG} Lock conflict [offload-l2] (task=${task.id}): ${lockKey}, skip (timer will re-trigger)`);
|
||||
const msgId = (task as any)._msgId;
|
||||
if (msgId) {
|
||||
try { await this.backend.ackTask(msgId); } catch { /* best effort */ }
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
// Lock conflict: current coroutine waits locally (no re-enqueue to stream).
|
||||
// Exponential backoff: 200ms → 600ms → 1.8s → 5s (capped), retry until lockTtlMs exhausted.
|
||||
// 旧版本固定 sleep(5000) 在 instance 级锁下会造成排队体感差(同 instance 多 session 累积秒级延迟);
|
||||
@@ -265,14 +324,14 @@ export class PipelineWorker {
|
||||
let delay = 200;
|
||||
while (Date.now() < deadline && this.running) {
|
||||
attempt++;
|
||||
this.logger?.debug?.(`${TAG} Lock conflict: ${lockKey}, retry ${attempt} after ${delay}ms`);
|
||||
this.logger?.debug?.(`${TAG} Lock conflict [${task.type}] (task=${task.id}): ${lockKey}, retry ${attempt} after ${delay}ms`);
|
||||
await this.sleep(delay);
|
||||
acquired = await this.backend.acquireLock(lockKey, this.config.workerId, this.config.lockTtlMs);
|
||||
if (acquired) break;
|
||||
delay = Math.min(delay * 3, 5000);
|
||||
}
|
||||
if (!acquired) {
|
||||
this.logger?.warn?.(`${TAG} Lock conflict timeout: ${lockKey}, dropping task`);
|
||||
this.logger?.warn?.(`${TAG} Lock conflict timeout [${task.type}] (task=${task.id}): ${lockKey}, dropping task`);
|
||||
// CR-1 fix: ACK to prevent stale recovery from re-claiming this message in an
|
||||
// infinite loop. Without it, XPENDING keeps returning this msgId every
|
||||
// pendingRecoveryIntervalMs, exhausting worker slots.
|
||||
@@ -396,6 +455,20 @@ export class PipelineWorker {
|
||||
this.activeLocks.delete(lockKey);
|
||||
this.runningTasks.delete(task.id);
|
||||
try { await this.backend.releaseLock(lockKey, this.config.workerId); } catch { /* best effort */ }
|
||||
|
||||
// Step 7: 延迟入队 — executor 可通过 task._deferredEnqueue 暂存需要在锁释放后才入队的任务,
|
||||
// 避免新任务立即被消费时因同 session 锁仍被持有而产生不必要的锁冲突。
|
||||
const deferred = (task as any)._deferredEnqueue as TaskPayload[] | undefined;
|
||||
if (deferred?.length) {
|
||||
for (const dTask of deferred) {
|
||||
try {
|
||||
await this.backend.enqueueTask(dTask);
|
||||
this.logger?.debug?.(`${TAG} Deferred enqueue: ${dTask.type} [${dTask.id}]`);
|
||||
} catch (err) {
|
||||
this.logger?.warn?.(`${TAG} Deferred enqueue failed: ${err instanceof Error ? err.message : String(err)}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -405,6 +478,9 @@ export class PipelineWorker {
|
||||
case "L2": return this.executor.executeL2(task, signal);
|
||||
case "L3": return this.executor.executeL3(task, signal);
|
||||
case "flush": return this.executor.executeFlush?.(task, signal) ?? this.executor.executeL1(task, signal);
|
||||
case "offload-l1": return this.executor.executeOffloadL1?.(task, signal);
|
||||
case "offload-l15": return this.executor.executeOffloadL15?.(task, signal);
|
||||
case "offload-l2": return this.executor.executeOffloadL2?.(task, signal);
|
||||
default:
|
||||
this.logger.warn(`${TAG} Unknown task type: ${task.type}`);
|
||||
}
|
||||
@@ -494,7 +570,22 @@ export class PipelineWorker {
|
||||
* 升级窗口期,新旧 pod 看到的 lock key 格式不同,可能短暂破坏跨 pod
|
||||
* 互斥。缓解: 升级时停所有 worker → 等 pending 任务清空 → 启动新版。
|
||||
*/
|
||||
private getLockKey(task: TaskPayload): string {
|
||||
private getLockKey(task: TaskPayload): string | null {
|
||||
// offload-l1 is lock-free: rename guarantees exclusive file ownership,
|
||||
// appendFile is atomic (O_APPEND), and state.json is read-only for L1.
|
||||
if (task.type === "offload-l1") return null;
|
||||
|
||||
// offload-l2: per-MMD lock so different MMDs can be processed concurrently.
|
||||
if (task.type === "offload-l2") {
|
||||
const mmdFile = (task.data as any)?.targetMmdFile ?? "default";
|
||||
return `pipeline:{${task.instanceId}}:offload-l2:${mmdFile}`;
|
||||
}
|
||||
|
||||
// offload-l15: lock-free at worker level. The executor acquires a short
|
||||
// lock only during the final write phase (state.json update), allowing
|
||||
// multiple L1.5 LLM calls to run concurrently without blocking each other.
|
||||
if (task.type === "offload-l15") return null;
|
||||
|
||||
if (this.config.lockGranularity === "instance") {
|
||||
return `pipeline:{${task.instanceId}}`;
|
||||
}
|
||||
|
||||
@@ -152,7 +152,7 @@ export class TimerScanner {
|
||||
}
|
||||
|
||||
for (const entry of expired) {
|
||||
const { instanceId, sessionId, taskType, priority } = this.parseShardMember(entry.member);
|
||||
const { instanceId, sessionId, taskType, priority, timerType } = this.parseShardMember(entry.member);
|
||||
|
||||
const task: TaskPayload = {
|
||||
id: `${taskType}-${instanceId.slice(-8)}-${sessionId.slice(-8)}-${now}`,
|
||||
@@ -161,7 +161,7 @@ export class TimerScanner {
|
||||
sessionId,
|
||||
priority,
|
||||
createdAt: now,
|
||||
data: { triggeredBy: "timer_scanner", timerMember: `${sessionId}:${taskType === "L1" ? "L1_idle" : taskType === "L2" ? "L2_schedule" : "L3"}`, instanceId },
|
||||
data: { triggeredBy: "timer_scanner", timerMember: `${sessionId}:${timerType}`, instanceId },
|
||||
};
|
||||
|
||||
await this.backend.enqueueTask(task);
|
||||
@@ -191,7 +191,7 @@ export class TimerScanner {
|
||||
* Parse shard member format: "{instanceId}\x00{sessionId}:{timerType}"
|
||||
* Example: "mem-j4wjesud\x00sess_001:L1_idle" → { instanceId: "mem-j4wjesud", sessionId: "sess_001", taskType: "L1" }
|
||||
*/
|
||||
private parseShardMember(member: string): { instanceId: string; sessionId: string; taskType: "L1" | "L2" | "L3" | "flush"; priority: number } {
|
||||
private parseShardMember(member: string): { instanceId: string; sessionId: string; taskType: TaskPayload["type"]; priority: number; timerType: string } {
|
||||
const sep = member.indexOf("\x00");
|
||||
let instanceId: string;
|
||||
let rest: string;
|
||||
@@ -206,19 +206,51 @@ export class TimerScanner {
|
||||
rest = member.slice(firstColon + 1);
|
||||
}
|
||||
|
||||
// rest = "sessionId:timerType" (e.g. "sess_001:L1_idle", "sess_001:L2_schedule")
|
||||
// rest = timer member after instanceId separator
|
||||
// New unified format: "offload-{type}:{embeddedInstanceId}:{sessionId}[:{extra}]" (prefix-based)
|
||||
// Legacy format: "sessionId:L1_idle" or "sessionId:L2_schedule"
|
||||
|
||||
// Check for offload prefix format first
|
||||
if (rest.startsWith("offload-l15:")) {
|
||||
// Skip embedded instanceId: "offload-l15:{instanceId}:{sessionId}"
|
||||
const afterPrefix = rest.slice("offload-l15:".length);
|
||||
const colonIdx = afterPrefix.indexOf(":");
|
||||
const sessionId = colonIdx > 0 ? afterPrefix.slice(colonIdx + 1) : afterPrefix;
|
||||
return { instanceId, sessionId, taskType: "offload-l15", priority: 0, timerType: rest };
|
||||
}
|
||||
if (rest.startsWith("offload-l2:")) {
|
||||
// Skip embedded instanceId: "offload-l2:{instanceId}:{sessionId}[:{mmdFile}]"
|
||||
const afterPrefix = rest.slice("offload-l2:".length);
|
||||
const colonIdx = afterPrefix.indexOf(":");
|
||||
let sessionId = colonIdx > 0 ? afterPrefix.slice(colonIdx + 1) : afterPrefix;
|
||||
// Strip trailing ":{mmdFile}" from sessionId
|
||||
if (sessionId.endsWith(".mmd")) {
|
||||
const lastColon = sessionId.lastIndexOf(":");
|
||||
if (lastColon > 0) sessionId = sessionId.slice(0, lastColon);
|
||||
}
|
||||
return { instanceId, sessionId, taskType: "offload-l2", priority: 1, timerType: rest };
|
||||
}
|
||||
if (rest.startsWith("offload-l1:")) {
|
||||
// Skip embedded instanceId: "offload-l1:{instanceId}:{sessionId}"
|
||||
const afterPrefix = rest.slice("offload-l1:".length);
|
||||
const colonIdx = afterPrefix.indexOf(":");
|
||||
const sessionId = colonIdx > 0 ? afterPrefix.slice(colonIdx + 1) : afterPrefix;
|
||||
return { instanceId, sessionId, taskType: "offload-l1", priority: 0, timerType: rest };
|
||||
}
|
||||
|
||||
// For non-offload types, use lastColon (original logic)
|
||||
const lastColon = rest.lastIndexOf(":");
|
||||
if (lastColon <= 0) {
|
||||
return { instanceId, sessionId: rest, taskType: "L1", priority: 0 };
|
||||
return { instanceId, sessionId: rest, taskType: "L1", priority: 0, timerType: "L1_idle" };
|
||||
}
|
||||
|
||||
const sessionId = rest.slice(0, lastColon);
|
||||
const timerType = rest.slice(lastColon + 1);
|
||||
|
||||
if (timerType.startsWith("L1")) return { instanceId, sessionId, taskType: "L1", priority: 0 };
|
||||
if (timerType.startsWith("L2")) return { instanceId, sessionId, taskType: "L2", priority: 1 };
|
||||
if (timerType.startsWith("L3")) return { instanceId, sessionId, taskType: "L3", priority: 2 };
|
||||
return { instanceId, sessionId, taskType: "flush", priority: 0 };
|
||||
if (timerType.startsWith("L1")) return { instanceId, sessionId, taskType: "L1", priority: 0, timerType };
|
||||
if (timerType.startsWith("L2")) return { instanceId, sessionId, taskType: "L2", priority: 1, timerType };
|
||||
if (timerType.startsWith("L3")) return { instanceId, sessionId, taskType: "L3", priority: 2, timerType };
|
||||
return { instanceId, sessionId, taskType: "flush", priority: 0, timerType };
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
* Guarantees:
|
||||
* 1. Blank conversation history (temporary session file)
|
||||
* 2. Independent system prompt (only the task prompt)
|
||||
* 3. No tool calls (tools restricted to minimal read-only set to avoid empty tools[] rejection by some providers)
|
||||
* 3. No tool calls when enableTools=false (disableTools:true — no tool definitions sent to API)
|
||||
* 4. No contamination from the main agent's context
|
||||
*/
|
||||
|
||||
@@ -17,6 +17,7 @@ import { fileURLToPath, pathToFileURL } from "node:url";
|
||||
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/core";
|
||||
import { getEnv } from "./env.js";
|
||||
import { report } from "../core/report/reporter.js";
|
||||
import type { Logger } from "../core/types.js";
|
||||
|
||||
/**
|
||||
* Resolve a preferred temporary directory for memory-tdai operations.
|
||||
@@ -49,12 +50,7 @@ function resolveOpenClawTmpDir(): string {
|
||||
|
||||
const TAG = "[memory-tdai] [runner]";
|
||||
|
||||
interface RunnerLogger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
type RunnerLogger = Logger;
|
||||
|
||||
// Dynamic import type — runEmbeddedPiAgent is an internal API
|
||||
// Prefer the public plugin runtime signature so host-injected runtimes stay assignable.
|
||||
@@ -394,10 +390,11 @@ export class CleanContextRunner {
|
||||
},
|
||||
tools: {
|
||||
...((this.options.config as Record<string, unknown>)?.tools as Record<string, unknown> | undefined),
|
||||
// When enableTools=false we still keep one lightweight read-only tool
|
||||
// so that the tools array sent to the API is non-empty.
|
||||
// Some providers (e.g. qwencode) reject tools:[] with minItems:1 validation.
|
||||
allow: this.options.enableTools ? ["read", "write", "edit"] : ["read"],
|
||||
// When enableTools=true, restrict to the minimal set needed for
|
||||
// scene extraction (read/write/edit).
|
||||
// When enableTools=false, pass an empty allow list — disableTools:true
|
||||
// will prevent tools from being sent to the API entirely.
|
||||
allow: this.options.enableTools ? ["read", "write", "edit"] : [],
|
||||
},
|
||||
// Override the full agent system prompt with the caller's extraction-specific
|
||||
// system prompt. This replaces OpenClaw's default system prompt (identity,
|
||||
@@ -455,11 +452,13 @@ export class CleanContextRunner {
|
||||
runId,
|
||||
provider: this.resolvedProvider,
|
||||
model: this.resolvedModel,
|
||||
// Do NOT pass disableTools:true — that produces tools:[] which some
|
||||
// providers (qwencode) reject with "[] is too short - 'tools'".
|
||||
// Instead rely on cleanConfig.tools.allow to restrict the tool set
|
||||
// to a minimal read-only tool (when enableTools=false).
|
||||
disableTools: false,
|
||||
// When enableTools=false, pass disableTools:true so that no tool
|
||||
// definitions are sent to the API. This avoids polluting the LLM
|
||||
// context with tool schemas and prevents the model from attempting
|
||||
// tool calls during pure text extraction tasks.
|
||||
// If a provider (e.g. qwencode) rejects empty tools[], users should
|
||||
// switch to StandaloneLLMRunner via LLM configuration instead.
|
||||
disableTools: !this.options.enableTools,
|
||||
extraSystemPrompt: effectiveSystemPrompt,
|
||||
streamParams: {
|
||||
maxTokens: params.maxTokens,
|
||||
|
||||
@@ -3,13 +3,7 @@ import path from "node:path";
|
||||
|
||||
import type { IMemoryStore } from "../core/store/types.js";
|
||||
import { ManagedTimer } from "./managed-timer.js";
|
||||
|
||||
interface Logger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
import type { Logger } from "../core/types.js";
|
||||
|
||||
export interface MemoryCleanerOptions {
|
||||
baseDir: string;
|
||||
|
||||
@@ -37,6 +37,7 @@ import { PersonaTrigger } from "../core/persona/persona-trigger.js";
|
||||
import { PersonaGenerator } from "../core/persona/persona-generator.js";
|
||||
import { pullProfilesToLocal, syncLocalProfilesToStore } from "../core/profile/profile-sync.js";
|
||||
import type { StorageAdapter } from "../core/storage/adapter.js";
|
||||
import type { Logger } from "../core/types.js";
|
||||
|
||||
const TAG = "[memory-tdai] [pipeline-factory]";
|
||||
|
||||
@@ -72,12 +73,8 @@ function supportsProfileSyncWrite(store?: IMemoryStore): boolean {
|
||||
// Logger interface
|
||||
// ============================
|
||||
|
||||
export interface PipelineLogger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
/** @deprecated Use `Logger` from `../core/types.js` directly. */
|
||||
export type PipelineLogger = Logger;
|
||||
|
||||
// ============================
|
||||
// Factory options
|
||||
|
||||
@@ -81,18 +81,12 @@ import { SessionFilter } from "./session-filter.js";
|
||||
import { ManagedTimer } from "./managed-timer.js";
|
||||
import { SerialQueue } from "./serial-queue.js";
|
||||
import { report } from "../core/report/reporter.js";
|
||||
import type { Logger } from "../core/types.js";
|
||||
|
||||
// ============================
|
||||
// Types
|
||||
// ============================
|
||||
|
||||
interface Logger {
|
||||
debug?: (message: string) => void;
|
||||
info: (message: string) => void;
|
||||
warn: (message: string) => void;
|
||||
error: (message: string) => void;
|
||||
}
|
||||
|
||||
/** A single captured message ready for L1 processing. */
|
||||
export interface CapturedMessage {
|
||||
role: "user" | "assistant" | "tool";
|
||||
|
||||
Reference in New Issue
Block a user