feat: release v1.0.0

This commit is contained in:
chrishuan
2026-06-09 15:28:05 +08:00
parent bc1575a0da
commit 9b7dbdd7b2
83 changed files with 7737 additions and 335 deletions
+67
View File
@@ -4,6 +4,71 @@
---
## [1.0.0] - 2026-06-11
> **正式版发布**:从 OpenClaw 专属插件演进为**面向所有 Agent 的通用记忆服务**。完整的 Gateway 独立服务 + v2 HTTP API + 官方 TypeScript / Python SDK,任何 Agent 框架均可接入完整的多层记忆与上下文压缩能力。
### ⚠️ Breaking Changes
- **客户端/服务端架构拆分**:记忆引擎从 OpenClaw 嵌入式插件拆分为独立 Gateway 服务进程,部署方式与接入方式发生变化。
- **配置结构变更**:插件配置结构扁平化重构,原 `gateway` 字段迁移为 `server` 嵌套。
- **插件入口模式变更**:支持 `local`(进程内本地运行,默认)和 `client`(连接外部 Memory Gateway)两种接入模式。
### ✨ 新功能
#### 独立 Gateway 服务(v2 API
记忆能力不再绑定 OpenClaw 宿主,以独立服务形式运行,通过 v2 HTTP API 为任意 Agent 提供记忆读写与管线管理:
- **完整 v2 API**14 条标准路由覆盖记忆 CRUD、原子更新、场景索引、管线状态查询等全部操作。
- **管线状态查询(`/v2/pipeline/status`**:实时获取 L1/L2/L3 各阶段运行状态与进度。
- **实例生命周期管理(`/v2/instance/destroy`)**:支持外部系统主动创建/销毁记忆实例。
- **可选 Bearer 鉴权 + CORS 白名单**:保护对外暴露的 API 安全。
- **请求体校验**:强制 1 MiB 上限,防止异常请求。
#### 官方 SDK
- **TypeScript SDK 1.0.0**`@tencentdb-agent-memory/memory-sdk-ts`):类型安全,覆盖全部 v2 API,npm 安装即用。
- **Python SDK**`tencentdb-agent-memory-sdk-python`):pip wheel 安装,同步/异步双模式,覆盖全部 v2 API。
#### 通用 Agent 框架适配
- **OpenClaw 插件适配**:支持 `local`(进程内本地运行,默认)和 `client`(连接外部 Memory Gateway)两种接入模式。`local` 模式保持原有体验。
- **Hermes Agent 适配**`memory_tencentdb_v2` adapter,支持 Hermes 框架多租户场景。
- **通用接入**:任何能发 HTTP 请求的 AgentLangChain、AutoGPT、自研框架等)均可通过 SDK 或裸 API 接入。
#### 可观测性
- **OpenTelemetry 全链路 Trace**:支持 OTLP 协议上报,可对接 Jaeger / Grafana Tempo 等后端。
- **Langfuse 集成**:仅转发 LLM 相关 span,适合评估记忆提取质量。
- **管线评测指标**:L1 提取率、去重决策分布、各阶段 token 消耗、recall 延迟等。
#### 部署
- **Standalone Docker 镜像**:单容器即可运行完整记忆服务,无需外部依赖。
- **Hermes + Memory All-in-One 镜像**Agent + 记忆服务一体化部署方案。
- **`memory-tencentdb-ctl` CLI**:运维命令行工具,支持配置管理、存储引擎切换等操作。
- **v2 安装器脚本**:一键配置插件适配层。
### 📖 文档
- SDK 接入指南(TypeScript / Python 示例)。
- Gateway 独立部署文档。
- OpenClaw / Hermes 插件安装说明。
- Docker 部署与配置参考。
---
<details>
<summary>1.0.0 预发布版本</summary>
## [1.0.0-beta.2] - 2026-06-05
### ✨ 新功能
- 统一插件入口,支持 local(进程内本地运行,默认)和 client(连接外部 Memory Gateway)两种接入模式。
- 新增 Offload Server V2L1/L1.5/L2 异步执行器、L3 压缩处理器、MMD 注入;Gateway 集成 /v2/offload/* 路由;
## [1.0.0-beta.1] - 2026-05-29
> **架构大版本**:从 OpenClaw 专属插件演进为**面向所有 Agent 的通用记忆服务**。核心能力通过独立 Gateway 服务 + 标准化 v2 HTTP API 对外暴露,配套官方 TypeScript / Python SDK,任何 Agent 框架均可接入完整的多层记忆管线。
@@ -56,6 +121,8 @@
- OpenClaw / Hermes 插件安装说明。
- Docker 部署与配置参考。
</details>
---
## [0.3.6] - 2026-05-27
+47
View File
@@ -121,6 +121,53 @@ docker pull agentmemory/hermes-memory:1.0.0-beta
docker pull agentmemory/openclaw-memory:1.0.0-beta
```
### 1.2 Zero-config to enable
Defaults to a local `SQLite + sqlite-vec` backend.
```jsonc
// ~/.openclaw/openclaw.json
{
"memory-tencentdb": {
"enabled": true
}
}
```
Once enabled, TencentDB Agent Memory automatically handles conversation capture, memory extraction, scene aggregation, persona generation, and recall before the next turn.
### 1.3 Enable short-term compression (optional, requires version ≥ 0.3.4)
```jsonc
{
"memory-tencentdb": {
"config": {
"offload": {
"enabled": true
}
}
}
}
```
#### Step 1 — Register the slot in your plugin config
Add the `slots` field so OpenClaw routes context-offload requests to this plugin:
```jsonc
{
"plugins": {
"slots": {
"contextEngine": "memory-tencentdb"
}
}
}
```
#### Step 2 — Apply the runtime patch
For the best results, run the patch script below. It hooks `after-tool-call` messages so they can be offloaded and recovered correctly:
#### Hermes + Memory (standalone)
```bash
+49
View File
@@ -1,3 +1,4 @@
<div align="center">
<img src="./assets/images/logo.png" alt="TencentDB Agent Memory" width="880" />
@@ -126,6 +127,54 @@ docker pull agentmemory/hermes-memory:1.0.0-beta
docker pull agentmemory/openclaw-memory:1.0.0-beta
```
### 1.2 零配置启用
默认使用本地 `SQLite + sqlite-vec` 后端。
```jsonc
// ~/.openclaw/openclaw.json
{
"memory-tencentdb": {
"enabled": true
}
}
```
启用后,TencentDB Agent Memory 会自动完成对话录制、记忆提取、场景归纳、用户画像生成和下一轮对话前召回。
### 1.3 启用短期记忆压缩(可选,要求版本 ≥ 0.3.4)
```jsonc
{
"memory-tencentdb": {
"config": {
"offload": {
"enabled": true
}
}
}
}
```
#### 步骤 1 —— 在插件配置中注册 slot
`slots` 字段中声明 `contextEngine`,让 OpenClaw 把上下文卸载请求路由到本插件:
```jsonc
{
"plugins": {
"slots": {
"contextEngine": "memory-tencentdb"
}
}
}
```
#### 步骤 2 —— 执行 patch 脚本
为保证最佳效果,请执行以下 patch 脚本。该脚本会注入 `after-tool-call` 消息钩子,让工具调用结果能被正确卸载与回溯:
#### Hermes + Memorystandalone
```bash
@@ -971,11 +971,19 @@ class MemoryTencentdbProvider(MemoryProvider):
except Exception as e:
logger.debug("memory-tencentdb session end failed: %s", e)
# Note: do NOT shut down the supervisor/Gateway here — it may serve
# other sessions. The Gateway manages its own lifecycle.
# We *do* drop our reference to the supervisor so any in-flight
# _try_recover_gateway() call sees self._supervisor is None and
# bails out instead of resurrecting a released provider.
# Stop only the Gateway process this supervisor spawned. If the
# provider merely attached to an already-running external Gateway,
# GatewaySupervisor.shutdown() is a no-op because _process is None.
supervisor = self._supervisor
if supervisor is not None:
try:
supervisor.shutdown()
except Exception as e:
logger.debug("memory-tencentdb supervisor shutdown failed: %s", e)
# Drop our reference so any in-flight _try_recover_gateway() call sees
# self._supervisor is None and bails out instead of resurrecting a
# released provider.
self._client = None
self._gateway_available = False
self._initialized = False
@@ -8,12 +8,22 @@ On shutdown(), sends a flush signal and waits for clean exit.
from __future__ import annotations
import contextlib
import logging
import os
import re
import shlex
import signal
import subprocess
import tempfile
import threading
import time
from typing import IO, Optional
from typing import Dict, IO, Iterator, Optional
try:
import fcntl
except ImportError: # pragma: no cover - Windows fallback
fcntl = None # type: ignore[assignment]
from .client import MemoryTencentdbSdkClient
@@ -31,6 +41,39 @@ HEALTH_CHECK_RETRIES = 3 # retries for is_running check
# Log file rotation parameters
LOG_TAIL_BYTES_ON_CRASH = 2048 # bytes of stderr log to surface on startup crash
# Startup single-flight state. The thread lock prevents multiple supervisor
# instances in the same Python process from spawning the same Gateway
# concurrently; the optional fcntl lock extends the guard across processes.
_START_LOCKS: Dict[str, threading.Lock] = {}
_START_LOCKS_GUARD = threading.Lock()
def _lock_key(host: str, port: int) -> str:
safe_host = re.sub(r"[^A-Za-z0-9_.-]", "_", host)
return f"{safe_host}-{port}"
@contextlib.contextmanager
def _startup_singleflight(host: str, port: int) -> Iterator[None]:
key = _lock_key(host, port)
with _START_LOCKS_GUARD:
thread_lock = _START_LOCKS.setdefault(key, threading.Lock())
lock_path = os.path.join(tempfile.gettempdir(), f"memory-tencentdb-gateway-{key}.lock")
with thread_lock:
lock_file = None
try:
if fcntl is not None:
lock_file = open(lock_path, "a", encoding="utf-8")
fcntl.flock(lock_file.fileno(), fcntl.LOCK_EX)
yield
finally:
if lock_file is not None:
try:
fcntl.flock(lock_file.fileno(), fcntl.LOCK_UN)
finally:
lock_file.close()
class GatewaySupervisor:
"""Manages the memory-tencentdb Gateway sidecar lifecycle."""
@@ -76,6 +119,7 @@ class GatewaySupervisor:
self._stdout_log: Optional[IO[bytes]] = None
self._stderr_log: Optional[IO[bytes]] = None
self._stderr_log_path: Optional[str] = None
self._shutdown_requested = False
# Resolve Gateway command
# Priority: explicit arg > MEMORY_TENCENTDB_GATEWAY_CMD env
@@ -146,77 +190,96 @@ class GatewaySupervisor:
logger.info("memory-tencentdb Gateway already running at %s", self._base_url)
return True
# If we previously spawned a child and it has since died, drop the
# stale Popen handle so the new spawn below isn't shadowed by a
# zombie reference. Without this, a crashed-then-respawned Gateway
# would keep ``self._process`` pointing at the dead PID forever and
# ``is_process_alive()`` would mislead the watchdog.
self._reap_dead_process()
# Try to start the Gateway
if not self._gateway_cmd:
logger.warning(
"memory-tencentdb Gateway is not running and no gateway command configured. "
"Set MEMORY_TENCENTDB_GATEWAY_CMD environment variable or pass gateway_cmd to supervisor. "
"memory-tencentdb memory will be unavailable."
)
return False
logger.info("Starting memory-tencentdb Gateway: %s", self._gateway_cmd)
try:
env = os.environ.copy()
env["MEMORY_TENCENTDB_GATEWAY_PORT"] = str(self._port)
env["MEMORY_TENCENTDB_GATEWAY_HOST"] = self._host
# Note: we deliberately do NOT inject TDAI_GATEWAY_API_KEY into
# the child's env from here. Whether the Gateway enforces auth is
# the operator's call — they configure it on the Gateway side
# (env, yaml, docker run, systemd unit) just like any other
# Gateway setting. The supervisor's ``api_key`` is purely the
# client-side Bearer token used for outbound requests.
# Redirect child stdout/stderr to log files instead of PIPE.
# Using PIPE without an active reader will deadlock the child once
# the pipe buffer (~64 KB) fills up. A log directory next to the
# data dir keeps logs inspectable on crash while eliminating the
# blocking risk entirely.
log_dir = self._resolve_log_dir()
try:
os.makedirs(log_dir, exist_ok=True)
except OSError as e:
logger.warning(
"memory-tencentdb Gateway: failed to create log dir %s (%s); "
"falling back to DEVNULL", log_dir, e,
with _startup_singleflight(self._host, self._port):
# Another supervisor may have started the Gateway while we were
# waiting on the single-flight lock. Re-probe before spawning.
if self.is_running():
logger.info(
"memory-tencentdb Gateway became available at %s while waiting for startup lock",
self._base_url,
)
log_dir = None
return True
if log_dir is not None:
stdout_path = os.path.join(log_dir, "gateway.stdout.log")
stderr_path = os.path.join(log_dir, "gateway.stderr.log")
# Append mode: preserve previous runs for postmortem.
self._stdout_log = open(stdout_path, "ab", buffering=0)
self._stderr_log = open(stderr_path, "ab", buffering=0)
self._stderr_log_path = stderr_path
stdout_target: object = self._stdout_log
stderr_target: object = self._stderr_log
else:
stdout_target = subprocess.DEVNULL
stderr_target = subprocess.DEVNULL
# If we previously spawned a child and it has since died, drop the
# stale Popen handle so the new spawn below isn't shadowed by a
# zombie reference. Without this, a crashed-then-respawned Gateway
# would keep ``self._process`` pointing at the dead PID forever and
# ``is_process_alive()`` would mislead the watchdog.
self._reap_dead_process()
self._process = subprocess.Popen(
shlex.split(self._gateway_cmd),
env=env,
stdout=stdout_target,
stderr=stderr_target,
start_new_session=True, # Detach from parent process group
)
except Exception as e:
logger.error("Failed to start memory-tencentdb Gateway: %s", e)
self._close_log_handles()
return False
# Try to start the Gateway
if not self._gateway_cmd:
logger.warning(
"memory-tencentdb Gateway is not running and no gateway command configured. "
"Set MEMORY_TENCENTDB_GATEWAY_CMD environment variable or pass gateway_cmd to supervisor. "
"memory-tencentdb memory will be unavailable."
)
return False
# Wait for health check
return self._wait_for_health()
logger.info("Starting memory-tencentdb Gateway: %s", self._gateway_cmd)
self._shutdown_requested = False
try:
env = os.environ.copy()
# The Python provider historically used MEMORY_TENCENTDB_* while
# src/gateway/config.ts reads TDAI_GATEWAY_*. Export both so a
# non-default supervisor port cannot accidentally spawn a child
# that still binds the default 8420.
env["MEMORY_TENCENTDB_GATEWAY_PORT"] = str(self._port)
env["MEMORY_TENCENTDB_GATEWAY_HOST"] = self._host
env["TDAI_GATEWAY_PORT"] = str(self._port)
env["TDAI_GATEWAY_HOST"] = self._host
# Note: we deliberately do NOT inject TDAI_GATEWAY_API_KEY into
# the child's env from here. Whether the Gateway enforces auth is
# the operator's call — they configure it on the Gateway side
# (env, yaml, docker run, systemd unit) just like any other
# Gateway setting. The supervisor's ``api_key`` is purely the
# client-side Bearer token used for outbound requests.
# Redirect child stdout/stderr to log files instead of PIPE.
# Using PIPE without an active reader will deadlock the child once
# the pipe buffer (~64 KB) fills up. A log directory next to the
# data dir keeps logs inspectable on crash while eliminating the
# blocking risk entirely.
log_dir = self._resolve_log_dir()
try:
os.makedirs(log_dir, exist_ok=True)
except OSError as e:
logger.warning(
"memory-tencentdb Gateway: failed to create log dir %s (%s); "
"falling back to DEVNULL", log_dir, e,
)
log_dir = None
if log_dir is not None:
stdout_path = os.path.join(log_dir, "gateway.stdout.log")
stderr_path = os.path.join(log_dir, "gateway.stderr.log")
# Append mode: preserve previous runs for postmortem.
self._stdout_log = open(stdout_path, "ab", buffering=0)
self._stderr_log = open(stderr_path, "ab", buffering=0)
self._stderr_log_path = stderr_path
stdout_target: object = self._stdout_log
stderr_target: object = self._stderr_log
else:
stdout_target = subprocess.DEVNULL
stderr_target = subprocess.DEVNULL
self._process = subprocess.Popen(
shlex.split(self._gateway_cmd),
env=env,
stdout=stdout_target,
stderr=stderr_target,
start_new_session=True,
)
except Exception as e:
logger.error("Failed to start memory-tencentdb Gateway: %s", e)
self._close_log_handles()
return False
# Keep the lock until the spawned process becomes healthy; otherwise
# a second waiter can observe the port as down during cold start and
# launch a duplicate process that immediately hits EADDRINUSE.
return self._wait_for_health()
def _resolve_log_dir(self) -> str:
"""Pick a directory to store Gateway stdout/stderr logs.
@@ -269,6 +332,10 @@ class GatewaySupervisor:
"""Wait for the Gateway to become healthy."""
start = time.monotonic()
while time.monotonic() - start < HEALTH_CHECK_MAX_WAIT:
if self._shutdown_requested:
logger.info("memory-tencentdb Gateway startup wait cancelled by shutdown")
return False
# Check if process died
if self._process and self._process.poll() is not None:
rc = self._process.returncode
@@ -303,20 +370,32 @@ class GatewaySupervisor:
def shutdown(self) -> None:
"""Shut down the managed Gateway process (if we started it)."""
self._shutdown_requested = True
if self._process is None:
return
logger.info("Shutting down memory-tencentdb Gateway...")
try:
# Send SIGTERM for graceful shutdown
self._process.terminate()
proc = self._process
if proc.poll() is None:
# The Gateway is started with start_new_session=True. Terminate
# the whole process group so `pnpm -> tsx -> node server.ts`
# does not leave the real listener orphaned after the top-level
# wrapper exits.
try:
os.killpg(os.getpgid(proc.pid), signal.SIGTERM)
except Exception:
proc.terminate()
try:
self._process.wait(timeout=10)
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
logger.warning("memory-tencentdb Gateway did not exit in 10s, sending SIGKILL")
self._process.kill()
self._process.wait(timeout=5)
try:
os.killpg(os.getpgid(proc.pid), signal.SIGKILL)
except Exception:
proc.kill()
proc.wait(timeout=5)
except Exception as e:
logger.warning("Error shutting down memory-tencentdb Gateway: %s", e)
finally:
@@ -231,6 +231,69 @@ def test_reap_dead_process_keeps_alive_handle():
assert sup._process is alive
def test_ensure_running_singleflight_prevents_duplicate_spawn(monkeypatch):
"""Concurrent supervisors for one port must spawn at most one Gateway."""
port = 28421
running = False
spawn_envs = []
spawn_lock = threading.Lock()
class FakePopen:
pid = 12345
returncode = None
def poll(self):
return None
def fake_is_running(self):
return running
def fake_wait_for_health(self):
nonlocal running
time.sleep(0.1)
running = True
return True
def fake_popen(_argv, *, env, stdout, stderr, start_new_session):
with spawn_lock:
spawn_envs.append(dict(env))
assert start_new_session is True
return FakePopen()
monkeypatch.setattr(supervisor_module.GatewaySupervisor, "is_running", fake_is_running)
monkeypatch.setattr(supervisor_module.GatewaySupervisor, "_wait_for_health", fake_wait_for_health)
monkeypatch.setattr(supervisor_module.subprocess, "Popen", fake_popen)
supervisors = [
supervisor_module.GatewaySupervisor(
host="127.0.0.1",
port=port,
gateway_cmd="fake gateway",
)
for _ in range(6)
]
barrier = threading.Barrier(len(supervisors))
results = []
def worker(sup):
barrier.wait()
results.append(sup.ensure_running())
threads = [threading.Thread(target=worker, args=(sup,)) for sup in supervisors]
for t in threads:
t.start()
for t in threads:
t.join(timeout=3)
assert results == [True] * len(supervisors)
assert len(spawn_envs) == 1
env = spawn_envs[0]
assert env["MEMORY_TENCENTDB_GATEWAY_PORT"] == str(port)
assert env["MEMORY_TENCENTDB_GATEWAY_HOST"] == "127.0.0.1"
assert env["TDAI_GATEWAY_PORT"] == str(port)
assert env["TDAI_GATEWAY_HOST"] == "127.0.0.1"
# ---------------------------------------------------------------------------
# Watchdog: detects death, resurrects, and reattaches
# ---------------------------------------------------------------------------
+49 -5
View File
@@ -22,6 +22,7 @@ import path from "node:path";
import fs from "node:fs";
import { createRequire } from "node:module";
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/core";
import registerClientOpenClawPlugin from "./openclaw-plugin/index.js";
import { parseConfig } from "./src/config.js";
import type { MemoryTdaiConfig } from "./src/config.js";
import { executeReadCos, READ_COS_TOOL_SCHEMA, READ_COS_TOOL_NAME, READ_COS_TOOL_DESCRIPTION } from "./src/core/tools/read-cos.js";
@@ -29,6 +30,7 @@ import { LocalStorageBackend } from "./src/core/storage/local-backend.js";
import { StaticCredentialProvider } from "./src/core/storage/credential-provider.js";
import type { IStorageBackend } from "./src/core/storage/types.js";
import { registerOffload } from "./src/offload/index.js";
import { registerOffloadClient } from "./src/offload-client/index.js";
import {
setPreferredEmbeddedAgentRuntime,
prewarmEmbeddedAgent,
@@ -52,6 +54,22 @@ import { resolveOpenClawStateDir } from "./src/utils/openclaw-state-dir.js";
const TAG = "[memory-tdai]";
type OpenClawAdapterMode = "local" | "client";
function resolveOpenClawAdapterMode(rawPluginConfig: Record<string, unknown> | undefined): OpenClawAdapterMode {
const rawMode = typeof rawPluginConfig?.mode === "string"
? rawPluginConfig.mode.trim().toLowerCase()
: "";
if (rawMode === "client" || rawMode === "gateway" || rawMode === "remote") {
return "client";
}
// Default to local/function mode: OpenClaw calls this plugin in-process and
// memory processing uses the host LLM runner (no standalone Gateway needed).
return "local";
}
/**
* Epoch ms when the plugin was registered (cold-start timestamp).
* Used as a fallback cursor in performAutoCapture when no checkpoint
@@ -158,6 +176,13 @@ export default function register(api: OpenClawPluginApi) {
}
// ─── Full / discovery mode: complete runtime initialization ───
const rawPluginConfigForMode = api.pluginConfig as Record<string, unknown> | undefined;
const adapterMode = resolveOpenClawAdapterMode(rawPluginConfigForMode);
if (adapterMode === "client") {
api.logger.info?.(`${TAG} mode=client: delegating to memory-tencentdb-client adapter`);
return registerClientOpenClawPlugin(api as any);
}
pluginStartTimestamp = Date.now();
setPreferredEmbeddedAgentRuntime(api.runtime.agent);
// Reset reporter singleton so config changes take effect on hot-reload.
@@ -189,7 +214,7 @@ export default function register(api: OpenClawPluginApi) {
`pipeline=(everyN=${cfg.pipeline.everyNConversations}, warmup=${cfg.pipeline.enableWarmup}, l1Idle=${cfg.pipeline.l1IdleTimeoutSeconds}s, l2DelayAfterL1=${cfg.pipeline.l2DelayAfterL1Seconds}s, l2Min=${cfg.pipeline.l2MinIntervalSeconds}s, l2Max=${cfg.pipeline.l2MaxIntervalSeconds}s, activeWindow=${cfg.pipeline.sessionActiveWindowHours}h), ` +
`persona(triggerEvery=${cfg.persona.triggerEveryN}, backupCount=${cfg.persona.backupCount}, sceneBackupCount=${cfg.persona.sceneBackupCount}), ` +
`memoryCleanup(enabled=${cfg.memoryCleanup.enabled}, retentionDays=${cfg.memoryCleanup.retentionDays ?? "(disabled)"}, cleanTime=${cfg.memoryCleanup.cleanTime}), ` +
`offload(enabled=${cfg.offload.enabled}, backendUrl=${cfg.offload.backendUrl ?? "(none)"}, mildRatio=${cfg.offload.mildOffloadRatio}, aggressiveRatio=${cfg.offload.aggressiveCompressRatio}, retentionDays=${cfg.offload.offloadRetentionDays})`,
`offload(enabled=${cfg.offload.enabled}, mode=${cfg.offload.mode}, ${cfg.offload.mode === "client" ? `serverUrl=${cfg.offload.serverUrl ?? "(none)"}` : `backendUrl=${cfg.offload.backendUrl ?? "(none)"}, mildRatio=${cfg.offload.mildOffloadRatio}, aggressiveRatio=${cfg.offload.aggressiveCompressRatio}`}, retentionDays=${cfg.offload.offloadRetentionDays})`,
);
} catch (err) {
api.logger.error(`${TAG} Config parsing failed: ${err instanceof Error ? err.message : String(err)}`);
@@ -232,7 +257,7 @@ export default function register(api: OpenClawPluginApi) {
try {
ensurePluginHookPolicy({
rootConfig: api.config,
runtimeConfig: api.runtime?.config,
runtimeConfig: (api.runtime as any)?.config,
logger: api.logger,
});
} catch (err) {
@@ -556,7 +581,10 @@ export default function register(api: OpenClawPluginApi) {
storagePromise = (async (): Promise<IStorageBackend> => {
if (cosSecretId && cosSecretKey && cosBucket) {
try {
const { CosStorageBackend } = await import("./src/integrations/cos/cos-backend.js");
const cosBackendModulePath = ["./src/integrations/cos", "cos-backend.js"].join("/");
const { CosStorageBackend } = await import(cosBackendModulePath) as {
CosStorageBackend: new (options: any) => IStorageBackend;
};
const backend = new CosStorageBackend({
credentialProvider: new StaticCredentialProvider({
secretId: cosSecretId,
@@ -954,8 +982,24 @@ export default function register(api: OpenClawPluginApi) {
if (cfg.offload.enabled) {
api.logger.debug?.(`${TAG} Offload enabled, registering offload module...`);
try {
registerOffload(api, cfg.offload);
api.logger.debug?.(`${TAG} Offload module registered successfully`);
if (cfg.offload.mode === "client") {
// New: stateless client mode — all compression delegated to server
registerOffloadClient(api as any, {
enabled: cfg.offload.enabled,
serverUrl: cfg.offload.serverUrl ?? "",
apiKey: cfg.offload.apiKey ?? "",
serviceId: cfg.offload.serviceId ?? "",
agentName: cfg.offload.agentName ?? "default",
compactionRatio: cfg.offload.compactionRatio ?? 0.5,
ingestTimeoutMs: cfg.offload.ingestTimeoutMs ?? 5000,
compactionTimeoutMs: cfg.offload.compactionTimeoutMs ?? 30000,
});
api.logger.debug?.(`${TAG} Offload client module registered (mode=client)`);
} else {
// Legacy: full local L3 compression
registerOffload(api, cfg.offload);
api.logger.debug?.(`${TAG} Offload module registered successfully (mode=${cfg.offload.mode})`);
}
} catch (err) {
api.logger.error(`${TAG} Offload module registration failed: ${err instanceof Error ? err.message : String(err)}`);
}
+38 -2
View File
@@ -2,7 +2,8 @@
* memory-tencentdb-client — OpenClaw 记忆插件(客户端接入版)
*
* 通过 @tencentdb-agent-memory/memory-sdk-ts 连接远端 memory server
* 提供四层记忆的自动捕获、召回和工具调用能力
* 提供四层记忆的自动捕获、召回和工具调用能力
* 并集成 offload 上下文压缩/摘要服务。
*
* 本插件不包含任何数据处理逻辑(无 VDB/Embedding/Pipeline),
* 所有操作委托给远端 server。
@@ -14,6 +15,8 @@ import { performCapture } from "./src/hooks/capture.js";
import { handleMemorySearch } from "./src/tools/memory-search.js";
import { handleConversationSearch } from "./src/tools/conversation-search.js";
import { handleReadCos } from "./src/tools/read-cos.js";
// @ts-ignore — rootDir mismatch in openclaw-plugin tsconfig; resolved at bundle time by tsdown
import { registerOffloadClient } from "../src/offload-client/index.js";
const TAG = "[memory-client]";
@@ -22,6 +25,7 @@ interface ServerConfig {
url?: string;
apiKey?: string;
instanceId?: string;
rejectUnauthorized?: boolean;
}
interface RecallConfig {
maxResults?: number;
@@ -31,10 +35,18 @@ interface RecallConfig {
interface CaptureConfig {
enabled?: boolean;
}
interface OffloadConfig {
enabled?: boolean;
serverUrl?: string;
compactionRatio?: number;
ingestTimeoutMs?: number;
compactionTimeoutMs?: number;
}
interface PluginConfig {
server?: ServerConfig;
recall?: RecallConfig;
capture?: CaptureConfig;
offload?: OffloadConfig;
}
// Matches OpenClaw plugin register() signature: export default function register(api)
@@ -44,6 +56,7 @@ export default function register(api: any) {
const server = cfg.server ?? {};
const recall = cfg.recall ?? {};
const capture = cfg.capture ?? {};
const offload = cfg.offload ?? {};
const serverUrl = server.url || "http://127.0.0.1:8420";
const apiKey = server.apiKey || "sk-xxxx";
@@ -52,6 +65,13 @@ export default function register(api: any) {
const includePersona = recall.includePersona !== false;
const includeSceneNav = recall.includeSceneNav !== false;
const captureEnabled = capture.enabled !== false;
const rejectUnauthorized = server.rejectUnauthorized !== false;
// Offload config (auto-inherits from server config)
const offloadEnabled = offload.enabled ?? false;
const offloadServerUrl = offload.serverUrl || serverUrl;
const offloadCompactionRatio = offload.compactionRatio ?? 0.5;
const offloadCompactionTimeoutMs = offload.compactionTimeoutMs ?? 30000;
// ── Initialize SDK ──
// NOTE: pass config (not a raw Transport) so client.readFile can lazily
@@ -60,12 +80,15 @@ export default function register(api: any) {
endpoint: serverUrl,
apiKey,
serviceId: instanceId,
rejectUnauthorized,
});
api.logger.info?.(
`${TAG} Initialized: server=${serverUrl}, instance=${instanceId}, ` +
`recall(persona=${includePersona},sceneNav=${includeSceneNav},max=${recallMaxResults}), ` +
`capture=${captureEnabled}`,
`capture=${captureEnabled}, offload=${offloadEnabled}, rejectUnauthorized=${rejectUnauthorized}`,
);
// ── Register Tools (same pattern as extensions/memory-tencentdb/index.ts) ──
@@ -249,4 +272,17 @@ export default function register(api: any) {
} else {
api.logger.info?.(`${TAG} capture disabled by config`);
}
// ── Offload: unified registration via registerOffloadClient ────────────────────
if (offloadEnabled) {
registerOffloadClient(api, {
enabled: true,
serverUrl: offloadServerUrl,
apiKey,
serviceId: instanceId,
agentName: "default",
compactionRatio: offloadCompactionRatio,
compactionTimeoutMs: offloadCompactionTimeoutMs,
});
}
}
+35
View File
@@ -35,6 +35,11 @@
"type": "string",
"default": "default",
"description": "Memory instance idHTTP header x-tdai-service-id"
},
"rejectUnauthorized": {
"type": "boolean",
"default": true,
"description": "是否校验 HTTPS 证书。默认 true;仅自签证书测试环境需要显式设为 false"
}
}
},
@@ -69,6 +74,36 @@
"description": "是否启用自动对话捕获 (L0)"
}
}
},
"offload": {
"type": "object",
"description": "Offload 压缩/摘要服务配置(可选,不配则复用 server 配置)",
"properties": {
"enabled": {
"type": "boolean",
"default": false,
"description": "是否启用 Offload 上下文压缩"
},
"serverUrl": {
"type": "string",
"description": "Offload server URL。不填则自动从 server.url 推导(替换端口为 9100"
},
"compactionRatio": {
"type": "number",
"default": 0.5,
"description": "上下文占比超过此阈值时触发压缩(0-1)"
},
"ingestTimeoutMs": {
"type": "number",
"default": 5000,
"description": "Ingest 请求超时(毫秒)"
},
"compactionTimeoutMs": {
"type": "number",
"default": 30000,
"description": "Compaction 请求超时(毫秒)"
}
}
}
}
}
+33 -14
View File
@@ -7,17 +7,32 @@
"onStartup": true
},
"contracts": {
"tools": ["tdai_memory_search", "tdai_conversation_search"]
"tools": ["tdai_memory_search", "tdai_conversation_search", "tdai_read_cos"]
},
"configSchema": {
"type": "object",
"additionalProperties": true,
"properties": {
"mode": {
"type": "string",
"enum": ["local", "function", "client", "gateway", "remote"],
"default": "local",
"description": "OpenClaw 接入模式:local/function(默认,本地进程内运行,使用 OpenClaw function/host LLM runner)或 client/gateway/remote(轻量客户端模式,连接外部 Memory Gateway"
},
"storeBackend": {
"type": "string",
"enum": ["sqlite", "tcvdb"],
"default": "sqlite",
"description": "存储后端:sqlite(本地 SQLite + sqlite-vec)或 tcvdb(腾讯云向量数据库)"
"description": "存储后端:sqlite(本地 SQLite + sqlite-vec)或 tcvdb(腾讯云向量数据库)。仅 local/function 模式生效"
},
"server": {
"type": "object",
"description": "client/gateway/remote 模式下的外部 Memory Gateway 连接配置",
"properties": {
"url": { "type": "string", "default": "http://127.0.0.1:8420", "description": "Memory Gateway URL" },
"apiKey": { "type": "string", "default": "local", "description": "API Key for Gateway authentication" },
"instanceId": { "type": "string", "default": "default", "description": "Memory instance idHTTP header x-tdai-service-id" }
}
},
"capture": {
"type": "object",
@@ -140,23 +155,27 @@
},
"offload": {
"type": "object",
"description": "Context Offload 设置 — 多层上下文压缩系统(独立开关,默认关闭)",
"description": "Context Offload 设置 — 多层上下文压缩系统(独立开关,默认关闭)。支持 local(本地 LLM 处理)和 client(复用 server.url 远端服务)两种模式",
"properties": {
"enabled": { "type": "boolean", "default": false, "description": "是否启用 Context Offload(默认关闭,不影响 Memory 功能)" },
"model": { "type": "string", "description": "Offload 使用的 LLM 模型(格式: provider/model),未填写时使用 openclaw 默认模型" },
"temperature": { "type": "number", "default": 0.2, "description": "LLM 温度参数" },
"forceTriggerThreshold": { "type": "number", "default": 4, "description": "累积多少个 tool pair 后强制触发 L1" },
"dataDir": { "type": "string", "description": "自定义数据目录(绝对路径),默认 ~/.openclaw/context-offload" },
"mode": { "type": "string", "enum": ["local", "backend", "client", "collect"], "description": "Offload 运行模式:local(本地 LLM 处理)、backend(走 backendUrl 远端 v1 服务)、client(复用 server.url 远端 offload v2 服务)、collect(仅数据采集,不压缩)。未设置时自动推导:有 backendUrl → backend,有 server.url → client,否则 → local" },
"model": { "type": "string", "description": "【local 模式】Offload 使用的 LLM 模型(格式: provider/model),未填写时使用 openclaw 默认模型" },
"temperature": { "type": "number", "default": 0.2, "description": "【local 模式】LLM 温度参数" },
"forceTriggerThreshold": { "type": "number", "default": 4, "description": "【local 模式】累积多少个 tool pair 后强制触发 L1" },
"dataDir": { "type": "string", "description": "【local 模式】自定义数据目录(绝对路径),默认 ~/.openclaw/context-offload" },
"defaultContextWindow": { "type": "number", "default": 200000, "description": "默认上下文窗口大小" },
"maxPairsPerBatch": { "type": "number", "default": 20, "description": "L1 每批最大 tool pair 数" },
"l2NullThreshold": { "type": "number", "default": 4, "description": "offload.jsonl 中 node_id=null 数量达到此阈值时触发 L2" },
"l2TimeoutSeconds": { "type": "number", "default": 300, "description": "距上次 L2 超过此秒数时触发 L2" },
"maxPairsPerBatch": { "type": "number", "default": 20, "description": "【local 模式】L1 每批最大 tool pair 数" },
"l2NullThreshold": { "type": "number", "default": 4, "description": "【local 模式】offload.jsonl 中 node_id=null 数量达到此阈值时触发 L2" },
"l2TimeoutSeconds": { "type": "number", "default": 300, "description": "【local 模式】距上次 L2 超过此秒数时触发 L2" },
"mildOffloadRatio": { "type": "number", "default": 0.5, "description": "温和压缩触发比例(占 context window" },
"aggressiveCompressRatio": { "type": "number", "default": 0.85, "description": "激进压缩触发比例" },
"mmdMaxTokenRatio": { "type": "number", "default": 0.2, "description": "MMD 注入 token 预算比例" },
"backendUrl": { "type": "string", "description": "后端服务 URL(如 https://offload-api.example.com),配置后 L1/L1.5/L2/L4 走后端" },
"backendApiKey": { "type": "string", "description": "后端 API 认证 token" },
"backendTimeoutMs": { "type": "number", "default": 10000, "description": "后端调用超时(毫秒)" }
"mmdMaxTokenRatio": { "type": "number", "default": 0.2, "description": "【local 模式】MMD 注入 token 预算比例" },
"compactionRatio": { "type": "number", "default": 0.5, "description": "【client 模式】上下文占比超过此阈值时触发压缩(0-1),复用 server.url 作为后端地址" },
"ingestTimeoutMs": { "type": "number", "default": 5000, "description": "【client 模式】Ingest 请求超时(毫秒)" },
"compactionTimeoutMs": { "type": "number", "default": 30000, "description": "【client 模式】Compaction 请求超时(毫秒)" },
"backendUrl": { "type": "string", "description": "【已废弃,请使用 mode=client】后端服务 URL,配置后 L1/L1.5/L2/L4 走后端" },
"backendApiKey": { "type": "string", "description": "【已废弃,请使用 mode=client】后端 API 认证 token" },
"backendTimeoutMs": { "type": "number", "default": 10000, "description": "【已废弃,请使用 mode=client】后端调用超时(毫秒)" }
}
}
}
+6 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@tencentdb-agent-memory/memory-tencentdb",
"version": "1.0.0-beta.1",
"version": "1.0.0",
"description": "Four-layer local memory system plugin for OpenClaw — auto-captures, structures, and profiles conversational knowledge using local LLM + SQLite vector search (L0→L1→L2→L3 pipeline)",
"type": "module",
"main": "./dist/index.mjs",
@@ -45,11 +45,14 @@
"scripts/read-local-memory/dist/",
"scripts/memory-tencentdb-ctl.sh",
"scripts/install_hermes_memory_tencentdb.sh",
"scripts/install-hermes-plugin-v2.sh",
"scripts/install-openclaw-plugin-v2.sh",
"scripts/README.memory-tencentdb-ctl.md",
"src",
"scripts/openclaw-after-tool-call-messages.patch.sh",
"scripts/setup-offload.sh",
"hermes-plugin/",
"openclaw-plugin/",
"openclaw.plugin.json",
"README.md",
"CHANGELOG.md",
@@ -90,6 +93,7 @@
"@opentelemetry/sdk-node": "^0.218.0",
"@opentelemetry/sdk-trace-base": "^2.7.1",
"@opentelemetry/semantic-conventions": "^1.41.1",
"@tencentdb-agent-memory/memory-sdk-ts": "^1.0.0",
"@tencentdb-agent-memory/tcvdb-text": "^0.1.1",
"ai": "^6.0.164",
"crc-32": "^1.2.2",
@@ -110,7 +114,7 @@
"cos-nodejs-sdk-v5": "^2.15.4",
"ioredis": "^5.10.1",
"kafkajs": "^2.2.4",
"opik": "^1.0.0"
"opik": "^1.11.14"
},
"peerDependencies": {
"node-llama-cpp": "^3.16.2",
-3
View File
@@ -121,9 +121,7 @@ if (fs.existsSync(file)) {
}
config.plugins ??= {};
config.plugins.slots ??= {};
config.plugins.entries ??= {};
config.plugins.slots.memory = pluginId;
const entry = config.plugins.entries[pluginId] ?? {};
entry.enabled = true;
@@ -173,7 +171,6 @@ cat >&2 <<EOF
[install-openclaw-plugin-v2] Done.
Memory plugin configured:
plugins.slots.memory = "$MEMORY_PLUGIN_ID"
plugins.entries["$MEMORY_PLUGIN_ID"].enabled = true
$HOOK_SUMMARY
server.url = "$TDAI_MEMORY_ENDPOINT"
+162
View File
@@ -0,0 +1,162 @@
#!/usr/bin/env bash
# ═══════════════════════════════════════════════════════════════════
# start-offload-local.sh — 本地启动 Offload Server(零外部依赖)
# ═══════════════════════════════════════════════════════════════════
#
# 使用 LocalStorageBackend(本地文件) + LocalStateBackend(进程内存)
# 替代生产环境的 COS + Redis,适合本地开发调试。
#
# 用法:
# bash scripts/start-offload-local.sh [start|stop|restart|status]
# 默认: start(后台运行)
#
# 环境变量(可选):
# PORT — 服务端口,默认 9100
# DATA_DIR — 数据存储目录,默认 /tmp/openclaw/data
# LOG_DIR — 日志输出目录,默认 /tmp/openclaw/logs
# API_KEY — 认证 API Key,默认 "dev-local-key"
# LLM_BASE_URL — LLM API 地址(L1/L1.5/L2 需要),默认空
# LLM_API_KEY — LLM API Key,默认空
# OPIK_ENABLED — 是否启用 Opik 追踪,默认 "false"
# OPIK_URL_OVERRIDE — Opik Server 地址
# OPIK_API_KEY — Opik API Key
# OPIK_WORKSPACE — Opik Workspace,默认 "default"
# OPIK_PROJECT_NAME — Opik Project,默认 "openclaw-offload-server"
# L2_NULL_THRESHOLD — L2 触发阈值,默认 6(设为 999999 可禁用 L2
# AGGRESSIVE_COMPRESS_RATIO — aggressive 压缩触发比例,默认 0.85(设为 999 可禁用)
# EMERGENCY_COMPRESS_RATIO — emergency 压缩触发比例,默认 0.95(设为 999 可禁用)
# ═══════════════════════════════════════════════════════════════════
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
# ── 配置 ──
PORT="${PORT:-9100}"
DATA_DIR="${DATA_DIR:-/tmp/openclaw/data}"
LOG_DIR="${LOG_DIR:-/tmp/openclaw/logs}"
API_KEY="${API_KEY:-dev-local-key}"
LLM_BASE_URL="${LLM_BASE_URL:-https://tokenhub.tencentmaas.com/v1}"
LLM_API_KEY="${LLM_API_KEY:-sk-yYBudowP30M9t4X99vBOtNzhqz4a39jud4i3Br328mh87qcr}"
LLM_MODEL="${LLM_MODEL:-ep-91sgogyz}"
OPIK_ENABLED="${OPIK_ENABLED:-true}"
OPIK_URL_OVERRIDE="${OPIK_URL_OVERRIDE:-http://21.6.226.49:5173/api}"
OPIK_API_KEY="${OPIK_API_KEY:-}"
OPIK_WORKSPACE="${OPIK_WORKSPACE:-default}"
OPIK_PROJECT_NAME="${OPIK_PROJECT_NAME:-offload_server_49}"
L2_NULL_THRESHOLD="${L2_NULL_THRESHOLD:-6}"
AGGRESSIVE_COMPRESS_RATIO="${AGGRESSIVE_COMPRESS_RATIO:-0.85}"
EMERGENCY_COMPRESS_RATIO="${EMERGENCY_COMPRESS_RATIO:-0.95}"
PID_FILE="/tmp/openclaw/offload.pid"
# ── 子命令 ──
CMD="${1:-start}"
do_stop() {
if [ -f "${PID_FILE}" ]; then
local pid
pid=$(cat "${PID_FILE}")
if kill -0 "${pid}" 2>/dev/null; then
kill "${pid}"
echo "[OK] Stopped (pid=${pid})"
else
echo "[WARN] Process ${pid} not running"
fi
rm -f "${PID_FILE}"
else
echo "[WARN] No PID file found"
fi
}
do_status() {
if [ -f "${PID_FILE}" ] && kill -0 "$(cat "${PID_FILE}")" 2>/dev/null; then
echo "[OK] Running (pid=$(cat "${PID_FILE}"), port=${PORT})"
else
echo "[OFF] Not running"
fi
}
do_start() {
# Kill existing if running
if [ -f "${PID_FILE}" ] && kill -0 "$(cat "${PID_FILE}")" 2>/dev/null; then
kill "$(cat "${PID_FILE}")" 2>/dev/null
sleep 1
fi
mkdir -p "${DATA_DIR}" "${LOG_DIR}"
local LOG_FILE="${LOG_DIR}/offload-server-$(date +%Y%m%d).log"
echo "═══════════════════════════════════════════════════════════"
echo " Offload Server — Local Mode (zero dependencies)"
echo "═══════════════════════════════════════════════════════════"
echo " Port: ${PORT}"
echo " Data: ${DATA_DIR}"
echo " Log: ${LOG_FILE}"
echo " API Key: ${API_KEY:0:8}..."
echo " LLM URL: ${LLM_BASE_URL}"
echo " LLM Model: ${LLM_MODEL}"
echo " Opik: ${OPIK_ENABLED} (${OPIK_URL_OVERRIDE:-disabled})"
echo " L2 threshold: ${L2_NULL_THRESHOLD}"
echo " Aggressive ratio: ${AGGRESSIVE_COMPRESS_RATIO}"
echo " Emergency ratio: ${EMERGENCY_COMPRESS_RATIO}"
echo "═══════════════════════════════════════════════════════════"
cd "${PROJECT_ROOT}"
# 动态生成临时配置文件,将 standalone yaml 与 offload 覆盖项合并
local OVERRIDE_CONFIG="/tmp/openclaw/tdai-gateway.override.yaml"
mkdir -p /tmp/openclaw
cat "${PROJECT_ROOT}/tdai-gateway.standalone.yaml" > "${OVERRIDE_CONFIG}"
cat >> "${OVERRIDE_CONFIG}" << YAML_EOF
# ── 动态注入的 offload 覆盖配置(由 start-offload-local.sh 生成)──
offload:
l2NullThreshold: ${L2_NULL_THRESHOLD}
aggressiveCompressRatio: ${AGGRESSIVE_COMPRESS_RATIO}
emergencyCompressRatio: ${EMERGENCY_COMPRESS_RATIO}
YAML_EOF
TDAI_GATEWAY_CONFIG="${OVERRIDE_CONFIG}" \
TDAI_GATEWAY_PORT="${PORT}" \
TDAI_GATEWAY_HOST="0.0.0.0" \
TDAI_DATA_DIR="${DATA_DIR}" \
STATE_BACKEND="local" \
TDAI_V2_API_KEY="${API_KEY}" \
TDAI_LLM_API_KEY="${LLM_API_KEY}" \
TDAI_LLM_BASE_URL="${LLM_BASE_URL}" \
TDAI_LLM_MODEL="${LLM_MODEL}" \
OPIK_ENABLED="${OPIK_ENABLED}" \
OPIK_URL_OVERRIDE="${OPIK_URL_OVERRIDE}" \
OPIK_API_KEY="${OPIK_API_KEY}" \
OPIK_WORKSPACE="${OPIK_WORKSPACE}" \
OPIK_PROJECT_NAME="${OPIK_PROJECT_NAME}" \
MEMORY_MAX_BODY_BYTES="10485760" \
nohup npx tsx src/gateway/server.ts >> "${LOG_FILE}" 2>&1 &
echo $! > "${PID_FILE}"
sleep 2
if kill -0 "$(cat "${PID_FILE}")" 2>/dev/null; then
echo "[OK] Started (pid=$(cat "${PID_FILE}"), log=${LOG_FILE})"
echo ""
echo " 验证: curl -s http://127.0.0.1:${PORT}/health"
echo " 停止: bash scripts/start-offload-local.sh stop"
echo " 日志: tail -f ${LOG_FILE}"
else
echo "[FAIL] Process exited, check log: ${LOG_FILE}"
tail -10 "${LOG_FILE}"
exit 1
fi
}
case "${CMD}" in
start) do_start ;;
stop) do_stop ;;
restart) do_stop; sleep 1; do_start ;;
status) do_status ;;
*)
echo "用法: bash scripts/start-offload-local.sh [start|stop|restart|status]"
exit 1
;;
esac
+20
View File
@@ -63,6 +63,23 @@ print(core["content"])
# L3: write core memory
client.write_core("# User Profile\n...")
# Offload v2: send tool pairs for server-side L1 async processing (fire-and-forget)
client.offload_ingest(
session_id="agent_sess_123",
tool_pairs=[
{"tool_name": "search", "tool_call_id": "call_1", "params": {"q": "..."}, "result": "...", "timestamp": "..."},
],
)
# Offload v2: server-side context compaction (sync wait for result)
compacted = client.offload_compact(
session_id="agent_sess_123",
messages=[...],
ratio=0.7,
context_window=128000,
)
print(compacted["messages"], compacted["report"])
# Read memory pipeline artifacts (e.g. persona.md, scene_blocks/*.md)
raw = client.read_file("scene_blocks/工作.md")
```
@@ -103,6 +120,9 @@ asyncio.run(main())
| L2 | `rm_scenario()` | `POST /v2/scenario/rm` |
| L3 | `read_core()` | `POST /v2/core/read` |
| L3 | `write_core()` | `POST /v2/core/write` |
| Offload | `offload_ingest()` | `POST /v2/offload/ingest` |
| Offload | `offload_compact()` | `POST /v2/offload/compact` |
| Offload | `offload_query_mmd()` | `POST /v2/offload/query-mmd` |
## Error Handling
+20
View File
@@ -63,6 +63,23 @@ print(core["content"])
# L3: 写入核心记忆
client.write_core("# User Profile\n...")
# Offload v2: 上报工具调用对,触发服务端 L1 异步处理(可 fire-and-forget
client.offload_ingest(
session_id="agent_sess_123",
tool_pairs=[
{"tool_name": "search", "tool_call_id": "call_1", "params": {"q": "..."}, "result": "...", "timestamp": "..."},
],
)
# Offload v2: 服务端上下文压缩(同步等待结果)
compacted = client.offload_compact(
session_id="agent_sess_123",
messages=[...],
ratio=0.7,
context_window=128000,
)
print(compacted["messages"], compacted["report"])
# 读取记忆 pipeline 产物(如 persona.md、scene_blocks/*.md
raw = client.read_file("scene_blocks/工作.md")
```
@@ -103,6 +120,9 @@ asyncio.run(main())
| L2 | `rm_scenario()` | `POST /v2/scenario/rm` |
| L3 | `read_core()` | `POST /v2/core/read` |
| L3 | `write_core()` | `POST /v2/core/write` |
| Offload | `offload_ingest()` | `POST /v2/offload/ingest` |
| Offload | `offload_compact()` | `POST /v2/offload/compact` |
| Offload | `offload_query_mmd()` | `POST /v2/offload/query-mmd` |
## 错误处理
+169
View File
@@ -255,6 +255,116 @@ class MemoryClient:
"""``POST /core/write``"""
return self._stub.post(f"{_V2}/core/write", {"content": content})
# -- Offload (Ingest + Compact + Query-MMD) ----------------------------
def offload_ingest(
self,
session_id: str,
tool_pairs: List[Dict[str, Any]],
*,
prompt: Optional[str] = None,
recent_messages: Optional[List[Dict[str, Any]]] = None,
) -> Dict[str, Any]:
"""``POST /v2/offload/ingest`` — 上报工具调用对,触发 L1 异步处理。
fire-and-forget 使用忽略返回值
Parameters
----------
session_id : str
会话 ID
tool_pairs : list[dict]
工具调用对列表每个元素包含 ``tool_name````tool_call_id``
``params````result````timestamp``可选 ``duration_ms``
prompt : str, optional
最新 user message用于 L1.5 任务判断
recent_messages : list[dict], optional
近期历史消息列表``role`` + ``content``辅助 L1 提取上下文
"""
return self._stub.post(
f"{_V2}/offload/ingest",
_strip_none({
"session_id": session_id,
"tool_pairs": tool_pairs,
"prompt": prompt,
"recent_messages": recent_messages,
}),
)
def offload_compact(
self,
session_id: str,
messages: List[Dict[str, Any]],
ratio: float,
total_tokens: int,
*,
context_window: Optional[int] = None,
message_tokens: Optional[List[int]] = None,
) -> Dict[str, Any]:
"""``POST /v2/offload/compact`` — 对 messages 执行服务端上下文压缩。
Parameters
----------
session_id : str
会话 ID
messages : list[dict]
当前完整对话消息列表
ratio : float
当前 token 使用比例已用 / context_window触发压缩策略判断
total_tokens : int
当前完整上下文的总 token 包含 system prompttool schemas 等不在
messages 中的隐性开销服务端用于计算 fixed overhead 和校准 token 估算
context_window : int, optional
模型 context window 大小token
message_tokens : list[int], optional
每条消息对应的 token 提供时可跳过服务端估算提升性能
Returns
-------
dict
``messages``压缩后消息列表+ ``report``压缩报告
"""
return self._stub.post(
f"{_V2}/offload/compact",
_strip_none({
"session_id": session_id,
"messages": messages,
"ratio": ratio,
"total_tokens": total_tokens,
"context_window": context_window,
"message_tokens": message_tokens,
}),
)
def offload_query_mmd(
self,
session_id: str,
*,
limit: Optional[int] = None,
) -> Dict[str, Any]:
"""``POST /v2/offload/query-mmd`` — 查询 session 的任务流程图(MMD 文件)。
Parameters
----------
session_id : str
会话 ID
limit : int, optional
最多返回几个 MMD 文件``limit=1`` 时走快速路径只返回当前活跃 MMD
Returns
-------
dict
``mmds``列表每项含 ``filename````content````version``+
``current_mmd``当前活跃 MMD 文件名无则为 ``None``
"""
return self._stub.post(
f"{_V2}/offload/query-mmd",
_strip_none({
"session_id": session_id,
"limit": limit,
}),
)
# -- File read (memory pipeline artifacts) -----------------------------
def read_file(self, path: str) -> str:
@@ -432,6 +542,65 @@ class AsyncMemoryClient:
async def write_core(self, content: str) -> Dict[str, Any]:
return await self._stub.post(f"{_V2}/core/write", {"content": content})
# -- Offload (Ingest + Compact + Query-MMD) ----------------------------
async def offload_ingest(
self,
session_id: str,
tool_pairs: List[Dict[str, Any]],
*,
prompt: Optional[str] = None,
recent_messages: Optional[List[Dict[str, Any]]] = None,
) -> Dict[str, Any]:
"""``POST /v2/offload/ingest``(异步)"""
return await self._stub.post(
f"{_V2}/offload/ingest",
_strip_none({
"session_id": session_id,
"tool_pairs": tool_pairs,
"prompt": prompt,
"recent_messages": recent_messages,
}),
)
async def offload_compact(
self,
session_id: str,
messages: List[Dict[str, Any]],
ratio: float,
total_tokens: int,
*,
context_window: Optional[int] = None,
message_tokens: Optional[List[int]] = None,
) -> Dict[str, Any]:
"""``POST /v2/offload/compact``(异步)"""
return await self._stub.post(
f"{_V2}/offload/compact",
_strip_none({
"session_id": session_id,
"messages": messages,
"ratio": ratio,
"total_tokens": total_tokens,
"context_window": context_window,
"message_tokens": message_tokens,
}),
)
async def offload_query_mmd(
self,
session_id: str,
*,
limit: Optional[int] = None,
) -> Dict[str, Any]:
"""``POST /v2/offload/query-mmd``(异步)"""
return await self._stub.post(
f"{_V2}/offload/query-mmd",
_strip_none({
"session_id": session_id,
"limit": limit,
}),
)
# -- lifecycle ---------------------------------------------------------
# -- File read (memory pipeline artifacts) -----------------------------
+25
View File
@@ -58,6 +58,28 @@ console.log(core.content);
// L3: write core memory
await client.writeCore({ content: "# User Profile\n..." });
// Offload v2: send tool pairs for server-side L1 async processing (fire-and-forget)
await client.offloadIngest({
session_id: "agent_sess_123",
tool_pairs: [
{ tool_name: "search", tool_call_id: "call_1", params: { q: "..." }, result: "...", timestamp: "..." },
],
});
// Offload v2: server-side context compaction (sync wait for result)
const compacted = await client.offloadCompact({
session_id: "agent_sess_123",
messages: [...],
ratio: 0.7,
context_window: 128000,
total_tokens: 160000,
});
console.log(compacted.messages, compacted.report);
// Offload v2: query MMD task graphs
const mmd = await client.offloadQueryMmd({ session_id: "agent_sess_123", limit: 1 });
console.log(mmd.current_mmd, mmd.mmds);
// Read memory pipeline artifacts (e.g. persona.md, scene_blocks/*.md)
const raw = await client.readFile("scene_blocks/工作.md");
```
@@ -80,6 +102,9 @@ const raw = await client.readFile("scene_blocks/工作.md");
| L2 | `rmScenario()` | `POST /v2/scenario/rm` |
| L3 | `readCore()` | `POST /v2/core/read` |
| L3 | `writeCore()` | `POST /v2/core/write` |
| Offload | `offloadIngest()` | `POST /v2/offload/ingest` |
| Offload | `offloadCompact()` | `POST /v2/offload/compact` |
| Offload | `offloadQueryMmd()` | `POST /v2/offload/query-mmd` |
## Error Handling
+25
View File
@@ -58,6 +58,28 @@ console.log(core.content);
// L3: 写入核心记忆
await client.writeCore({ content: "# User Profile\n..." });
// Offload v2: 上报工具调用对,触发服务端 L1 异步处理(fire-and-forget
await client.offloadIngest({
session_id: "agent_sess_123",
tool_pairs: [
{ tool_name: "search", tool_call_id: "call_1", params: { q: "..." }, result: "...", timestamp: "..." },
],
});
// Offload v2: 服务端上下文压缩(同步等待结果)
const compacted = await client.offloadCompact({
session_id: "agent_sess_123",
messages: [...],
ratio: 0.7,
context_window: 128000,
total_tokens: 160000,
});
console.log(compacted.messages, compacted.report);
// Offload v2: 查询任务流程图(MMD
const mmd = await client.offloadQueryMmd({ session_id: "agent_sess_123", limit: 1 });
console.log(mmd.current_mmd, mmd.mmds);
// 读取记忆 pipeline 产物(如 persona.md、scene_blocks/*.md
const raw = await client.readFile("scene_blocks/工作.md");
```
@@ -80,6 +102,9 @@ const raw = await client.readFile("scene_blocks/工作.md");
| L2 | `rmScenario()` | `POST /v2/scenario/rm` |
| L3 | `readCore()` | `POST /v2/core/read` |
| L3 | `writeCore()` | `POST /v2/core/write` |
| Offload | `offloadIngest()` | `POST /v2/offload/ingest` |
| Offload | `offloadCompact()` | `POST /v2/offload/compact` |
| Offload | `offloadQueryMmd()` | `POST /v2/offload/query-mmd` |
## 错误处理
+32
View File
@@ -25,6 +25,12 @@ import type {
CoreFile,
CoreWriteData,
CoreWriteRequest,
OffloadCompactData,
OffloadCompactRequest,
OffloadIngestData,
OffloadIngestRequest,
OffloadQueryMmdData,
OffloadQueryMmdRequest,
ScenarioFile,
ScenarioListData,
ScenarioListRequest,
@@ -148,6 +154,32 @@ export class MemoryClient {
return this.http.post(`${V2}/core/write`, params as unknown as Record<string, unknown>);
}
// -- Offload (Compaction + Ingest) ------------------------------------
/**
* Send tool pairs (+ optional context) to offload server for L1 processing.
* Fire-and-forget usage: caller can `.catch()` without blocking.
*/
offloadIngest(params: OffloadIngestRequest): Promise<OffloadIngestData> {
return this.http.post(`${V2}/offload/ingest`, stripUndefined(params as unknown as Record<string, unknown>));
}
/**
* Request server-side context compaction.
* Returns compacted messages + report, or throws on failure.
*/
offloadCompact(params: OffloadCompactRequest): Promise<OffloadCompactData> {
return this.http.post(`${V2}/offload/compact`, stripUndefined(params as unknown as Record<string, unknown>));
}
/**
* Query MMD task graphs for a session.
* limit=1 returns only the current active MMD (fast path).
*/
offloadQueryMmd(params: OffloadQueryMmdRequest): Promise<OffloadQueryMmdData> {
return this.http.post(`${V2}/offload/query-mmd`, stripUndefined(params as unknown as Record<string, unknown>));
}
// -- File read (memory pipeline artifacts) ----------------------------
/**
+25 -1
View File
@@ -6,4 +6,28 @@ export { MemoryClient, type MemoryClientConfig, type Transport } from "./client.
export { TDAMError } from "./errors.js";
export { HttpTransport, type HttpTransportOptions } from "./http.js";
export { MemoryFileReader, StsCredentialManager, StsCredential, createMemoryFileReader, cosV5Sign, type MemoryFileReaderConfig } from "./cos.js";
export type * from "./types.js";
export type {
// L0
ConversationItem, ConversationAddRequest, ConversationAddData,
ConversationQueryRequest, ConversationQueryData,
ConversationSearchRequest, ConversationSearchData,
ConversationDeleteRequest, ConversationDeleteData,
// L1
AtomicDetail, AtomicUpdateRequest, AtomicUpdateData,
AtomicQueryRequest, AtomicQueryData,
AtomicSearchRequest, AtomicSearchData,
AtomicDeleteRequest, AtomicDeleteData,
// L2
ScenarioEntry, ScenarioListRequest, ScenarioListData,
ScenarioReadRequest, ScenarioFile,
ScenarioWriteRequest, ScenarioWriteData,
ScenarioRmRequest,
// L3
CoreFile, CoreWriteRequest, CoreWriteData,
// Offload
OffloadToolPair, OffloadRecentMessage,
OffloadIngestRequest, OffloadIngestData,
OffloadCompactRequest, OffloadCompactData, OffloadCompactReport,
// Common
ApiResponseEnvelope,
} from "./types.js";
+72
View File
@@ -188,3 +188,75 @@ export interface CoreWriteRequest {
export interface CoreWriteData {
updated_at: string;
}
// ---------------------------------------------------------------------------
// Offload (Compaction + Ingest)
// ---------------------------------------------------------------------------
export interface OffloadToolPair {
tool_name: string;
tool_call_id: string;
params: unknown;
result: unknown;
error?: string;
timestamp: string;
duration_ms?: number;
}
export interface OffloadRecentMessage {
role: "user" | "assistant";
content: string;
}
export interface OffloadIngestRequest {
session_id: string;
tool_pairs: OffloadToolPair[];
prompt?: string;
recent_messages?: OffloadRecentMessage[];
}
export interface OffloadIngestData {
accepted: boolean;
}
export interface OffloadCompactRequest {
session_id: string;
messages: unknown[];
ratio: number;
context_window: number;
total_tokens: number;
message_tokens?: number[];
}
export interface OffloadCompactReport {
resolvedLevel: string;
originalCount: number;
compactedCount: number;
fastPathReplaced: number;
fastPathDeleted: number;
mildReplacements: number;
aggressiveDeleted: number;
emergencyDeleted: number;
mmdInjected: number;
}
export interface OffloadCompactData {
messages: unknown[];
report: OffloadCompactReport;
}
export interface OffloadQueryMmdRequest {
session_id: string;
limit?: number;
}
export interface OffloadMmdFile {
filename: string;
content: string;
version: number;
}
export interface OffloadQueryMmdData {
mmds: OffloadMmdFile[];
current_mmd: string | null;
}
+38 -4
View File
@@ -213,11 +213,12 @@ export interface OffloadConfig {
* LLM execution mode for L1/L1.5/L2 tasks.
* - "local": call LLM directly via AI SDK (uses offload.model or main agent model)
* - "backend": route through remote backend service (requires backendUrl)
* - "client": stateless client mode all compression delegated to offload server v2
* - "collect": data collection only runs L1/L1.5/L2 asynchronously but disables
* L3 compression and does NOT occupy the contextEngine slot (uses legacy compaction)
* Default: "local" (auto-detects based on backendUrl presence for backward compat)
*/
mode: "local" | "backend" | "collect";
mode: "local" | "backend" | "client" | "collect";
/** LLM model for offload tasks, format: "provider/model-id". Falls back to agents.defaults.model when omitted. */
model?: string;
/** LLM temperature (default: 0.2) */
@@ -265,6 +266,22 @@ export interface OffloadConfig {
* primary non-loopback IPv4 address.
*/
userId?: string;
// ── Client mode fields (used when mode === "client") ──────────────
/** Offload server v2 base URL (e.g. "http://localhost:9100"). */
serverUrl?: string;
/** Bearer token for offload server v2 Authorization header. */
apiKey?: string;
/** X-TDAI-Service-Id header value. */
serviceId?: string;
/** Agent name used in storage path (default: "default"). */
agentName?: string;
/** Client-side threshold: skip compaction when ratio < this value (default: 0.5). */
compactionRatio?: number;
/** Ingest request timeout in ms (default: 5000). */
ingestTimeoutMs?: number;
/** Compaction request timeout in ms (default: 30000). */
compactionTimeoutMs?: number;
}
/** Fully resolved plugin configuration (v3). */
@@ -439,10 +456,20 @@ export function parseConfig(raw: Record<string, unknown> | undefined): MemoryTda
// --- Offload ---
const offloadGroup = obj(c, "offload");
const offloadMode: "local" | "backend" | "collect" = (() => {
// Auto-derive offload serverUrl/apiKey/serviceId from top-level server config
// when offload client fields are not explicitly set.
const serverGroup = obj(c, "server");
const serverUrl = optStr(serverGroup, "url");
const serverApiKey = optStr(serverGroup, "apiKey");
const serverInstanceId = optStr(serverGroup, "instanceId");
const offloadMode: "local" | "backend" | "client" | "collect" = (() => {
const raw = optStr(offloadGroup, "mode");
if (raw === "local" || raw === "backend" || raw === "collect") return raw;
return optStr(offloadGroup, "backendUrl") ? "backend" : "local";
if (raw === "local" || raw === "backend" || raw === "client" || raw === "collect") return raw;
// Auto-derive: if backendUrl is set → "backend"; if server.url is set → "client"; else "local"
if (optStr(offloadGroup, "backendUrl")) return "backend";
if (serverUrl) return "client";
return "local";
})();
const offload: OffloadConfig = {
@@ -465,6 +492,13 @@ export function parseConfig(raw: Record<string, unknown> | undefined): MemoryTda
offloadRetentionDays: normalizeOffloadRetentionDays(num(offloadGroup, "offloadRetentionDays") ?? 0),
logMaxSizeMb: num(offloadGroup, "logMaxSizeMb") ?? 50,
userId: optStr(offloadGroup, "userId"),
// Client mode fields — fall back to top-level server config when not explicitly set
serverUrl: optStr(offloadGroup, "serverUrl") ?? serverUrl,
apiKey: optStr(offloadGroup, "apiKey") ?? serverApiKey,
serviceId: optStr(offloadGroup, "serviceId") ?? serverInstanceId,
compactionRatio: num(offloadGroup, "compactionRatio") ?? 0.5,
ingestTimeoutMs: num(offloadGroup, "ingestTimeoutMs") ?? 5000,
compactionTimeoutMs: num(offloadGroup, "compactionTimeoutMs") ?? 30000,
};
return {
+1 -7
View File
@@ -18,6 +18,7 @@ import crypto from "node:crypto";
import { sanitizeText, stripCodeBlocks, shouldCaptureL0 } from "../../utils/sanitize.js";
import type { StorageAdapter } from "../storage/adapter.js";
import { StoragePaths } from "../storage/types.js";
import type { Logger } from "../types.js";
// ============================
// Types
@@ -63,13 +64,6 @@ export interface L0ConversationRecord {
messages: ConversationMessage[];
}
interface Logger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
const TAG = "[memory-tdai][l0]";
// ============================
+2 -7
View File
@@ -20,14 +20,9 @@ import type { IMemoryStore, L0Record } from "../store/types.js";
import type { EmbeddingService } from "../store/embedding.js";
import type { StorageAdapter } from "../storage/adapter.js";
const TAG = "[memory-tdai] [capture]";
import type { Logger } from "../types.js";
interface Logger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
const TAG = "[memory-tdai] [capture]";
export interface AutoCaptureResult {
/** Whether the scheduler was notified (conversation count incremented) */
+8 -10
View File
@@ -21,6 +21,7 @@ import type { EmbeddingService, EmbeddingCallOptions } from "../store/embedding.
import { sanitizeText } from "../../utils/sanitize.js";
import type { StorageAdapter } from "../storage/adapter.js";
import { StoragePaths } from "../storage/types.js";
import type { Logger } from "../types.js";
const TAG = "[memory-tdai] [recall]";
const RECALL_TRUNCATION_SUFFIX = "…(已截断;可用 tdai_memory_search 或 tdai_conversation_search 查看详情)";
@@ -46,13 +47,6 @@ const MEMORY_TOOLS_GUIDE = `<memory-tools-guide>
- 3
</memory-tools-guide>`
interface Logger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
/** A single recalled L1 memory with its search score and type. */
export interface RecalledMemory {
content: string;
@@ -890,11 +884,15 @@ function normalizeBudgetLimit(value: number | undefined): number | undefined {
}
function truncateRecallLine(line: string, maxChars: number): string {
if (line.length <= maxChars) return line;
// Count and slice by code point, not UTF-16 code unit, so a cut never lands
// between the halves of a surrogate pair (which would corrupt a non-BMP
// character to U+FFFD when the line is UTF-8 encoded for the request).
const cps = Array.from(line);
if (cps.length <= maxChars) return line;
if (maxChars <= RECALL_TRUNCATION_SUFFIX.length) {
return line.slice(0, maxChars);
return cps.slice(0, maxChars).join("");
}
return `${line.slice(0, maxChars - RECALL_TRUNCATION_SUFFIX.length).trimEnd()}${RECALL_TRUNCATION_SUFFIX}`;
return `${cps.slice(0, maxChars - RECALL_TRUNCATION_SUFFIX.length).join("").trimEnd()}${RECALL_TRUNCATION_SUFFIX}`;
}
/**
+1 -8
View File
@@ -12,19 +12,12 @@ import { BackupManager } from "../../utils/backup.js";
import { escapeXmlTags } from "../../utils/sanitize.js";
import { report } from "../report/reporter.js";
import { reportL3LatencyMetrics } from "../report/metric-tracking-l3-latency.js";
import type { LLMRunner } from "../types.js";
import type { LLMRunner, Logger } from "../types.js";
import type { StorageAdapter } from "../storage/adapter.js";
import { StoragePaths } from "../storage/types.js";
const TAG = "[memory-tdai] [persona]";
interface Logger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
export class PersonaGenerator {
private dataDir: string;
private runner: LLMRunner;
+3 -6
View File
@@ -8,14 +8,11 @@ import { stripSceneNavigation } from "../scene/scene-navigation.js";
import type { StorageAdapter } from "../storage/adapter.js";
import { StoragePaths } from "../storage/types.js";
import type { Logger } from "../types.js";
const TAG = "[memory-tdai] [trigger]";
interface TriggerLogger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
type TriggerLogger = Logger;
export interface TriggerResult {
should: boolean;
+1 -7
View File
@@ -6,6 +6,7 @@ import { readSceneIndex, syncSceneIndex } from "../scene/scene-index.js";
import { generateSceneNavigation, stripSceneNavigation } from "../scene/scene-navigation.js";
import type { StorageAdapter } from "../storage/adapter.js";
import { StoragePaths } from "../storage/types.js";
import type { Logger } from "../types.js";
const PROFILE_SCOPE = "global";
@@ -15,13 +16,6 @@ function isRenameRaceError(err: unknown): boolean {
return code === "ENOTEMPTY" || code === "EEXIST";
}
interface Logger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
export interface ProfileBaseline {
version: number;
contentMd5: string;
+1 -8
View File
@@ -19,14 +19,7 @@ import { sanitizeJsonForParse } from "../../utils/sanitize.js";
import type { IMemoryStore } from "../store/types.js";
import { buildFtsQuery } from "../store/sqlite.js";
import type { EmbeddingService } from "../store/embedding.js";
import type { LLMRunner } from "../types.js";
interface Logger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
import type { LLMRunner, Logger } from "../types.js";
const TAG = "[memory-tdai][l1-dedup]";
+1 -8
View File
@@ -24,16 +24,9 @@ import type { EmbeddingService } from "../store/embedding.js";
import { report } from "../report/reporter.js";
import { metricProducer } from "../report/kafka-metric-producer.js";
import { reportL1LatencyMetrics } from "../report/metric-tracking-l1-latency.js";
import type { LLMRunner } from "../types.js";
import type { LLMRunner, Logger } from "../types.js";
import type { StorageAdapter } from "../storage/adapter.js";
interface Logger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
const TAG = "[memory-tdai][l1-extractor]";
// ============================
+1 -7
View File
@@ -19,13 +19,7 @@ import { StoragePaths } from "../storage/types.js";
// Re-export types that readers need
export type { MemoryRecord, MemoryType, EpisodicMetadata } from "./l1-writer.js";
export type { L1QueryFilter } from "../store/types.js";
interface Logger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
import type { Logger } from "../types.js";
const TAG = "[memory-tdai] [l1-reader]";
+1 -7
View File
@@ -21,6 +21,7 @@ import type { IMemoryStore } from "../store/types.js";
import type { EmbeddingService } from "../store/embedding.js";
import type { StorageAdapter } from "../storage/adapter.js";
import { StoragePaths } from "../storage/types.js";
import type { Logger } from "../types.js";
// ============================
// Types
@@ -110,13 +111,6 @@ export interface DedupDecision {
merged_timestamps?: string[];
}
interface Logger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
const TAG = "[memory-tdai][l1-writer]";
// ============================
+7 -2
View File
@@ -198,14 +198,19 @@ export class MetricTrackingRunner implements LLMRunner {
}
async run(params: LLMRunParams): Promise<string> {
// 0. 注入 instanceId(如果调用方没传,从 getInstanceId 回调获取)
const enrichedParams = params.instanceId
? params
: { ...params, instanceId: this.getInstanceId() };
// 1. 先执行原方法,拿到结果(异常直接 re-throw)
const text = await this.inner.run(params);
const text = await this.inner.run(enrichedParams);
// 2. 原方法成功后,try-catch 做上报(静默失败)
try {
const metricName = taskIdToMetricName(params.taskId);
if (metricName) {
const instanceId = params.instanceId ?? this.getInstanceId();
const instanceId = enrichedParams.instanceId ?? this.getInstanceId();
if (instanceId) {
// 优先从 inner runner 的 lastUsage side-channel 读取精确 token 数
const innerWithUsage = this.inner as LLMRunnerWithUsage;
+18
View File
@@ -62,6 +62,24 @@ export class TracedTaskExecutor implements TaskExecutor {
return this.executeL1(task);
}
async executeOffloadL1?(task: TaskPayload, signal?: AbortSignal): Promise<void> {
if (this.inner.executeOffloadL1) {
return this.executeWithTrace("offload-l1", task, () => this.inner.executeOffloadL1!(task, signal));
}
}
async executeOffloadL15?(task: TaskPayload, signal?: AbortSignal): Promise<void> {
if (this.inner.executeOffloadL15) {
return this.executeWithTrace("offload-l15", task, () => this.inner.executeOffloadL15!(task, signal));
}
}
async executeOffloadL2?(task: TaskPayload, signal?: AbortSignal): Promise<void> {
if (this.inner.executeOffloadL2) {
return this.executeWithTrace("offload-l2", task, () => this.inner.executeOffloadL2!(task, signal));
}
}
/**
* Trace Context
*
+19 -24
View File
@@ -27,18 +27,13 @@ import { normalizeSceneFilenames } from "./filename-normalizer.js";
import { buildSceneExtractionPrompt } from "../prompts/scene-extraction.js";
import { report } from "../report/reporter.js";
import { reportL2LatencyMetrics } from "../report/metric-tracking-l2-latency.js";
import type { LLMRunner } from "../types.js";
import type { LLMRunner, Logger } from "../types.js";
import type { StorageAdapter } from "../storage/adapter.js";
import { StoragePaths } from "../storage/types.js";
const TAG = "[memory-tdai] [extractor]";
interface ExtractorLogger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
type ExtractorLogger = Logger;
export interface ExtractionResult {
memoriesProcessed: number;
@@ -470,24 +465,24 @@ export class SceneExtractor {
success: true,
error: null,
});
}
// ── 评测指标:L2 延迟 + 场景变化 ──
try {
const postSceneCount = (await readSceneIndex(this.dataDir, this.storage)).length;
reportL2LatencyMetrics({
instanceId: this.instanceId ?? "",
extractionLatencyMs: totalMs,
llmDurationMs: llmDurationMs > 0 ? llmDurationMs : null,
sceneCountBefore: preExtractIndex.size,
sceneCountAfter: postSceneCount,
scenesCreated,
scenesUpdated,
scenesDeleted,
hasError: false,
});
} catch {
// 静默忽略
// ── 评测指标:L2 延迟 + 场景变化 ──
try {
const postSceneCount = (await readSceneIndex(this.dataDir, this.storage)).length;
reportL2LatencyMetrics({
instanceId: this.instanceId ?? "",
extractionLatencyMs: totalMs,
llmDurationMs: llmDurationMs > 0 ? llmDurationMs : null,
sceneCountBefore: preExtractIndex.size,
sceneCountAfter: postSceneCount,
scenesCreated,
scenesUpdated,
scenesDeleted,
hasError: false,
});
} catch {
// 静默忽略
}
}
// Detect empty extraction: pre and post scene_index both empty means LLM didn't write anything
+1 -1
View File
@@ -56,7 +56,7 @@ export interface TimerEntry {
export interface TaskPayload {
id: string;
type: "L1" | "L2" | "L3" | "flush";
type: "L1" | "L2" | "L3" | "flush" | "offload-l1" | "offload-l15" | "offload-l2";
instanceId: string;
sessionId: string;
priority: number; // 0=high, 1=normal, 2=low
+2 -7
View File
@@ -13,6 +13,8 @@
* check health to dynamically downgrade to pure semantic search.
*/
import type { Logger } from "../types.js";
// ============================
// Types
// ============================
@@ -20,13 +22,6 @@
/** Sparse vector: array of [token_hash, weight] pairs. */
export type SparseVector = Array<[number, number]>;
interface Logger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
export interface BM25ClientConfig {
/** Sidecar service URL (default: "http://127.0.0.1:8084") */
serviceUrl: string;
+1 -7
View File
@@ -11,16 +11,10 @@
import { BM25Encoder } from "@tencentdb-agent-memory/tcvdb-text";
import type { SparseVector } from "@tencentdb-agent-memory/tcvdb-text";
import type { Logger } from "../types.js";
export type { SparseVector };
interface Logger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
export interface BM25LocalConfig {
/** Whether BM25 sparse encoding is enabled (default: true) */
enabled: boolean;
+2 -11
View File
@@ -13,6 +13,8 @@
* - Throws on failure; callers decide fallback strategy.
*/
import type { Logger } from "../types.js";
// ============================
// Types
// ============================
@@ -104,17 +106,6 @@ export class EmbeddingNotReadyError extends Error {
}
}
// ============================
// Logger interface
// ============================
interface Logger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
const TAG = "[memory-tdai][embedding]";
// ============================
+11 -7
View File
@@ -21,6 +21,8 @@
*/
import { createRequire } from "node:module";
import { mkdirSync, existsSync } from "node:fs";
import path from "node:path";
import type { DatabaseSync, StatementSync, SQLInputValue } from "node:sqlite";
import type { MemoryRecord } from "../record/l1-writer.js";
import type { EmbeddingProviderInfo } from "./embedding.js";
@@ -40,6 +42,9 @@ import type {
L1PaginatedFilter,
L1PaginatedResult,
} from "./types.js";
import type { Logger } from "../types.js";
export type { L1RecordRow } from "./types.js";
// ============================
// Types
@@ -86,13 +91,6 @@ export interface L0RecordRow {
timestamp: number;
}
interface Logger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
const TAG = "[memory-tdai][sqlite]";
/** Persisted metadata about the embedding provider used to generate stored vectors. */
@@ -394,6 +392,12 @@ export class VectorStore implements IMemoryStore {
this.dimensions = dimensions;
this.logger = logger;
// Ensure parent directory exists (for non-default instance paths)
const dbDir = path.dirname(dbPath);
if (!existsSync(dbDir)) {
mkdirSync(dbDir, { recursive: true });
}
// Open database with extension support enabled
const { DatabaseSync: DbSync } = requireNodeSqlite();
this.db = new DbSync(dbPath, { allowExtension: true });
+6
View File
@@ -16,6 +16,7 @@
*/
import path from "node:path";
import { existsSync, mkdirSync } from "node:fs";
import type { MemoryTdaiConfig } from "../../config.js";
import type { IMemoryStore, StoreLogger } from "./types.js";
import type { EmbeddingService } from "./embedding.js";
@@ -303,6 +304,11 @@ export class StorePool {
const dims = embCfg.dimensions ?? 0;
const dbPath = this.getSqlitePath(instanceId);
// 确保数据库目录存在(对于非 default instance
const dbDir = path.dirname(dbPath);
if (!existsSync(dbDir)) {
mkdirSync(dbDir, { recursive: true });
}
const store = new VectorStore(dbPath, dims, this.logger as StoreLogger);
return {
+2 -6
View File
@@ -17,6 +17,7 @@
import type { MemoryRecord } from "../record/l1-writer.js";
import type { EmbeddingProviderInfo } from "./embedding.js";
import type { Logger } from "../types.js";
// Re-export so consumers can import everything from types.ts
export type { MemoryRecord, EmbeddingProviderInfo };
@@ -26,12 +27,7 @@ export type { MemoryRecord, EmbeddingProviderInfo };
// ============================
/** Minimal logger interface accepted by store implementations. */
export interface StoreLogger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
export type StoreLogger = Logger;
// ============================
// L1 Types (Structured Memories)
+1 -7
View File
@@ -13,18 +13,12 @@
import type { IMemoryStore, L0SearchResult } from "../store/types.js";
import { buildFtsQuery } from "../store/sqlite.js";
import type { EmbeddingService } from "../store/embedding.js";
import type { Logger } from "../types.js";
// ============================
// Types
// ============================
interface Logger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
export interface ConversationSearchResultItem {
id: string;
session_key: string;
+1 -7
View File
@@ -13,18 +13,12 @@
import type { IMemoryStore, L1SearchResult } from "../store/types.js";
import { buildFtsQuery } from "../store/sqlite.js";
import type { EmbeddingService } from "../store/embedding.js";
import type { Logger } from "../types.js";
// ============================
// Types
// ============================
interface Logger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
export interface MemorySearchResultItem {
id: string;
content: string;
+3 -3
View File
@@ -15,10 +15,10 @@
// ============================
/**
* Minimal logger interface used throughout TDAI Core.
* Canonical logger interface used across all TDAI modules.
*
* Matches the existing `StoreLogger` and `RunnerLogger` interfaces
* already used in the codebase no migration needed for existing callers.
* Named variants (StoreLogger, PluginLogger, etc.) are type aliases
* of this interface, kept for backward compatibility.
*/
export interface Logger {
debug?: (message: string) => void;
+41
View File
@@ -292,6 +292,25 @@ export interface GatewayConfig {
cos: CosExtraConfig;
/** 可观测性配置 (yaml: observability, env: KAFKA_METRIC_*) */
observability: ObservabilityConfig;
/** Offload server executor 配置 (yaml: offload) */
offload: {
forceTriggerThreshold: number;
pendingMaxAgeSeconds: number;
l1Temperature: number;
l1MaxTokens: number;
l1TimeoutMs: number;
l15Temperature: number;
l15MaxTokens: number;
l15TimeoutMs: number;
l2Temperature: number;
l2MaxTokens: number;
l2TimeoutMs: number;
l2NullThreshold: number;
mildOffloadRatio: number;
aggressiveCompressRatio: number;
emergencyCompressRatio: number;
maxRetries: number;
};
}
// ============================
@@ -491,6 +510,27 @@ export function loadGatewayConfig(overrides?: Partial<GatewayConfig>): GatewayCo
const observability: ObservabilityConfig = { otel, clickhouse, kafka, langfuse };
// Offload executor config (yaml: offload)
const offloadConfig = obj(fileConfig, "offload");
const offload = {
forceTriggerThreshold: num(offloadConfig, "forceTriggerThreshold") ?? 4,
pendingMaxAgeSeconds: num(offloadConfig, "pendingMaxAgeSeconds") ?? 30,
l1Temperature: num(offloadConfig, "l1Temperature") ?? 0.3,
l1MaxTokens: num(offloadConfig, "l1MaxTokens") ?? 8000,
l1TimeoutMs: num(offloadConfig, "l1TimeoutMs") ?? 120_000,
l15Temperature: num(offloadConfig, "l15Temperature") ?? 0.2,
l15MaxTokens: num(offloadConfig, "l15MaxTokens") ?? 3000,
l15TimeoutMs: num(offloadConfig, "l15TimeoutMs") ?? 120_000,
l2Temperature: num(offloadConfig, "l2Temperature") ?? 0.4,
l2MaxTokens: num(offloadConfig, "l2MaxTokens") ?? 16000,
l2TimeoutMs: num(offloadConfig, "l2TimeoutMs") ?? 120_000,
l2NullThreshold: num(offloadConfig, "l2NullThreshold") ?? 6,
mildOffloadRatio: num(offloadConfig, "mildOffloadRatio") ?? 0.5,
aggressiveCompressRatio: num(offloadConfig, "aggressiveCompressRatio") ?? 0.85,
emergencyCompressRatio: num(offloadConfig, "emergencyCompressRatio") ?? 0.95,
maxRetries: num(offloadConfig, "maxRetries") ?? 3,
};
const base: GatewayConfig = {
deployMode,
stateBackend,
@@ -505,6 +545,7 @@ export function loadGatewayConfig(overrides?: Partial<GatewayConfig>): GatewayCo
worker,
cos,
observability,
offload,
};
// Merge overrides one level deep so partial `server`/`data`/`llm` patches
+32
View File
@@ -46,6 +46,7 @@ export interface ClassifiedError {
*
* Recognized:
* - PayloadTooLargeError (CR-7) detected via duck-typing on statusCode === 413
* - Invalid JSON body parseJsonBody throws Error("Invalid JSON body") on malformed input
* - RecallFailure (H-15) code from RecallError taxonomy
* - SeedValidationError 400 with generic message
* - Anything else 500 Internal server error
@@ -64,6 +65,37 @@ export function classifyError(err: unknown): ClassifiedError {
};
}
// 1b. Invalid JSON body / decompression error — parseJsonBody throws this fixed message on
// JSON.parse failure or gzip/deflate decompression failure. Client error, not server fault.
if (err instanceof Error && err.message === "Invalid JSON body") {
return {
status: 400,
client: { code: 400, message: "Invalid JSON body", trace_id, retryable: false },
logLine: `[${trace_id}] InvalidJsonBody: request body could not be parsed as JSON`,
};
}
// 1c. COS AppendPositionErr — concurrent append conflict. The client should retry.
// This is a transient conflict, not a permanent error.
if (err instanceof Error && /AppendPositionErr|Position not equal object length/i.test(err.message)) {
return {
status: 409,
client: { code: 409, message: "Concurrent write conflict, please retry", trace_id, retryable: true },
logLine: `[${trace_id}] CosAppendConflict: ${err.message}`,
};
}
// 1d. Unsupported Content-Encoding — parseJsonBody rejects with this message when the
// client sends an encoding the server does not support (not gzip/deflate/identity).
if (err instanceof Error && err.message.startsWith("Unsupported Content-Encoding:")) {
const safeMsg = err.message.replace(/[^\w\s:/-]/g, "");
return {
status: 415,
client: { code: 415, message: safeMsg, trace_id, retryable: false },
logLine: `[${trace_id}] UnsupportedEncoding: ${err.message}`,
};
}
// 2. RecallFailure (H-15) — the recall layer has already produced a safe message.
// (Gateway handlers normally translate this into RecallResult.error before reaching
// the global catch, but if a code path forgets, this fallback ensures no raw leak.)
+201 -15
View File
@@ -17,6 +17,7 @@
import http from "node:http";
import { URL } from "node:url";
import { timingSafeEqual } from "node:crypto";
import zlib from "node:zlib";
import dayjs from "dayjs";
import { TdaiCore } from "../core/tdai-core.js";
import { StandaloneHostAdapter } from "../adapters/standalone/host-adapter.js";
@@ -53,6 +54,9 @@ import { executeSeed } from "../core/seed/seed-runtime.js";
import type { SeedProgress } from "../core/seed/types.js";
import { handleV2Route, errorEnvelope, makeRequestId } from "./v2-router.js";
import type { V2RouterDeps } from "./v2-router.js";
import { handleOffloadV2Route } from "../offload_server/router.js";
import type { OffloadV2Deps } from "../offload_server/router.js";
import { initServerOpikTracer } from "../offload_server/opik-tracer.js";
import { classifyError } from "./error-handler.js";
import { LocalStorageBackend } from "../core/storage/local-backend.js";
import { StorageAdapter } from "../core/storage/adapter.js";
@@ -142,11 +146,25 @@ export async function parseJsonBody<T>(req: http.IncomingMessage): Promise<T> {
return;
}
// Determine if the body is compressed (Content-Encoding header).
// Support gzip and deflate; reject unsupported encodings with 400.
const encoding = (req.headers["content-encoding"] ?? "").toLowerCase().trim();
let source: NodeJS.ReadableStream = req;
if (encoding === "gzip" || encoding === "x-gzip") {
source = req.pipe(zlib.createGunzip());
} else if (encoding === "deflate") {
source = req.pipe(zlib.createInflate());
} else if (encoding !== "" && encoding !== "identity") {
req.resume(); // drain
reject(new Error(`Unsupported Content-Encoding: ${encoding}`));
return;
}
const chunks: Buffer[] = [];
let received = 0;
let aborted = false;
req.on("data", (chunk: Buffer) => {
source.on("data", (chunk: Buffer) => {
if (aborted) return;
received += chunk.length;
if (received > MAX_BODY_BYTES) {
@@ -157,18 +175,19 @@ export async function parseJsonBody<T>(req: http.IncomingMessage): Promise<T> {
}
chunks.push(chunk);
});
req.on("end", () => {
source.on("end", () => {
if (aborted) return;
try {
const body = Buffer.concat(chunks).toString("utf-8");
resolve(JSON.parse(body) as T);
} catch (err) {
} catch {
reject(new Error("Invalid JSON body"));
}
});
req.on("error", (err) => {
source.on("error", (_err) => {
if (aborted) return; // already rejected with PayloadTooLargeError
reject(err);
// Decompression errors (e.g. truncated gzip) are client-side faults
reject(new Error("Invalid JSON body"));
});
});
}
@@ -333,6 +352,9 @@ export class TdaiGateway {
// Initialize core
await this.core.initialize();
// ── Initialize Opik tracer for offload server ──
await initServerOpikTracer(this.logger);
// ── Initialize StorageAdapter for v2 API ──
// In standalone mode, use LocalStorageBackend pointing to dataDir.
// In service mode, CosStorageBackend is injected externally.
@@ -578,6 +600,17 @@ export class TdaiGateway {
}
}
// ── Offload V2 routes (async ingest + mmd query) ──
const offloadDeps: OffloadV2Deps = {
resolveStorage: v2Deps.resolveStorage,
getStorage: v2Deps.getStorage ?? (() => undefined),
logger: this.logger,
stateBackend: this.stateBackend,
config: { ...this.config.offload, l1Model: "", l15Model: "", l2Model: "" },
};
const offloadHandled = await handleOffloadV2Route(req, res, pathname, method, parseJsonBody, sendJson, offloadDeps);
if (offloadHandled) return;
const handled = await handleV2Route(req, res, pathname, method, parseJsonBody, sendJson, v2Deps);
if (handled) return;
@@ -1101,26 +1134,65 @@ export class TdaiGateway {
type: backendType,
local: backendType === "local" ? {
onTimerExpired: (entry) => {
// Parse timer member: "sessionId:L2_schedule" or "sessionId:L1_idle"
const parts = entry.member.split(":");
const sessionId = parts.slice(0, -1).join(":");
const timerType = parts[parts.length - 1];
const taskType = timerType === "L2_schedule" ? "L2" : timerType === "L1_idle" ? "L1" : "L3";
const instanceId = this.config.instanceId ?? "default";
// Parse timer member by prefix: "offload-{type}:{instanceId}:{sessionId}[:{extra}]"
// or legacy "session:L2_schedule"
const member = entry.member;
let taskType: string;
let instanceId: string;
let sessionId: string;
const firstColon = member.indexOf(":");
const prefix = firstColon > 0 ? member.slice(0, firstColon) : member;
if (prefix === "offload-l1" || prefix === "offload-l15" || prefix === "offload-l2") {
taskType = prefix;
// Format: "offload-{type}:{instanceId}:{sessionId}[:{mmdFile}]"
// instanceId is the segment right after the prefix
const rest = member.slice(firstColon + 1);
const instanceEnd = rest.indexOf(":");
if (instanceEnd > 0) {
instanceId = rest.slice(0, instanceEnd);
sessionId = rest.slice(instanceEnd + 1);
} else {
instanceId = this.config.instanceId ?? "default";
sessionId = rest;
}
// For offload-l2: strip trailing ":{mmdFile}" from sessionId
// (mmdFile is extracted separately from timerMember in the executor)
if (prefix === "offload-l2" && sessionId.endsWith(".mmd")) {
const lastColon = sessionId.lastIndexOf(":");
if (lastColon > 0) {
sessionId = sessionId.slice(0, lastColon);
}
}
} else {
// Legacy format: "sessionId:L2_schedule" or "sessionId:L1_idle"
const lastColon = member.lastIndexOf(":");
const suffix = lastColon >= 0 ? member.slice(lastColon + 1) : "";
sessionId = lastColon >= 0 ? member.slice(0, lastColon) : member;
taskType = suffix === "L2_schedule" ? "offload-l2" : suffix === "L1_idle" ? "offload-l1" : "L3";
instanceId = this.config.instanceId ?? "default";
}
const now = Date.now();
// Extract targetMmdFile from member for offload-l2 (needed by pipeline-worker lockKey)
let targetMmdFile: string | undefined;
if (taskType === "offload-l2") {
const mmdMatch = member.match(/(\d+-[^:]+\.mmd)$/);
if (mmdMatch) targetMmdFile = mmdMatch[1];
}
const task = {
id: `${taskType}-${sessionId}-${now}`,
type: taskType,
type: taskType as any,
instanceId,
sessionId,
priority: 0,
createdAt: now,
data: { triggeredBy: "timer_scanner", timerMember: entry.member, instanceId },
data: { triggeredBy: "timer_scanner", timerMember: member, instanceId, targetMmdFile },
};
this.stateBackend!.enqueueTask(task).then(() => {
this.logger.info(`[local-timer] Timer fired: ${entry.member} → enqueued ${taskType} task`);
this.logger.info(`[local-timer] Timer fired: ${member} → enqueued ${taskType} task`);
}).catch((err) => {
this.logger.error(`[local-timer] Failed to enqueue task for ${entry.member}: ${err instanceof Error ? err.message : String(err)}`);
this.logger.error(`[local-timer] Failed to enqueue task for ${member}: ${err instanceof Error ? err.message : String(err)}`);
});
},
} : undefined,
@@ -1571,6 +1643,120 @@ export class TdaiGateway {
async executeFlush(task: TaskPayload) {
await core.handleSessionEnd(task.sessionId);
},
// ── Offload executors (L1 summary, L1.5 task judgment, L2 MMD update) ──
async executeOffloadL1(task: TaskPayload, signal?: AbortSignal) {
if (signal?.aborted) return;
const { OffloadTaskExecutor } = await import("../offload_server/offload-task-executor.js");
const storage = await resolveStorage(task);
if (!storage) return;
const llmClient = gateway.buildOffloadLlmClient();
if (!llmClient) {
gateway.logger.warn(`[executor] offload-l1 skipped: no LLM client available`);
return;
}
const executor = new OffloadTaskExecutor({
resolveStorage: async () => storage,
llmClient,
stateBackend: gateway.stateBackend!,
config: { ...gateway.config.offload, l1Model: "", l15Model: "", l2Model: "" },
logger: gateway.logger,
});
await executor.executeOffloadL1(task, signal);
},
async executeOffloadL15(task: TaskPayload, signal?: AbortSignal) {
if (signal?.aborted) return;
const { OffloadTaskExecutor } = await import("../offload_server/offload-task-executor.js");
const storage = await resolveStorage(task);
if (!storage) return;
const llmClient = gateway.buildOffloadLlmClient();
if (!llmClient) {
gateway.logger.warn(`[executor] offload-l15 skipped: no LLM client available`);
return;
}
const executor = new OffloadTaskExecutor({
resolveStorage: async () => storage,
llmClient,
stateBackend: gateway.stateBackend!,
config: { ...gateway.config.offload, l1Model: "", l15Model: "", l2Model: "" },
logger: gateway.logger,
});
await executor.executeOffloadL15(task, signal);
},
async executeOffloadL2(task: TaskPayload, signal?: AbortSignal) {
if (signal?.aborted) return;
const { OffloadTaskExecutor } = await import("../offload_server/offload-task-executor.js");
const storage = await resolveStorage(task);
if (!storage) return;
const llmClient = gateway.buildOffloadLlmClient();
if (!llmClient) {
gateway.logger.warn(`[executor] offload-l2 skipped: no LLM client available`);
return;
}
const executor = new OffloadTaskExecutor({
resolveStorage: async () => storage,
llmClient,
stateBackend: gateway.stateBackend!,
config: { ...gateway.config.offload, l1Model: "", l15Model: "", l2Model: "" },
logger: gateway.logger,
});
await executor.executeOffloadL2(task, signal);
},
};
}
/**
* Build a simple LLM client for offload executors using gateway's LLM config.
*/
private buildOffloadLlmClient() {
const llmCfg = this.config.llm;
if (!llmCfg.baseUrl || !llmCfg.apiKey || !llmCfg.model) return null;
const logger = this.logger;
return {
async chat(params: {
model: string;
messages: Array<{ role: "system" | "user"; content: string }>;
temperature: number;
max_tokens: number;
timeoutMs?: number;
}): Promise<string> {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), params.timeoutMs ?? 30000);
try {
const response = await fetch(`${llmCfg.baseUrl}/chat/completions`, {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${llmCfg.apiKey}`,
},
body: JSON.stringify({
model: llmCfg.model || params.model,
messages: params.messages,
temperature: params.temperature,
max_tokens: params.max_tokens,
}),
signal: controller.signal,
});
clearTimeout(timer);
if (!response.ok) {
throw new Error(`LLM API returned ${response.status}: ${await response.text()}`);
}
const json = (await response.json()) as any;
const finishReason = json.choices?.[0]?.finish_reason;
if (finishReason === "length") {
const content = json.choices?.[0]?.message?.content ?? "";
logger.warn(
`[offload-llm] Response truncated (finish_reason=length, max_tokens=${params.max_tokens}), ` +
`content=${content.length} chars`,
);
}
return json.choices?.[0]?.message?.content ?? "";
} catch (err) {
clearTimeout(timer);
throw err;
}
},
};
}
}
+526
View File
@@ -0,0 +1,526 @@
/**
* offload-client OffloadContextEngine.
* Occupies the Context Engine slot and delegates compression to the server.
*/
import type { OffloadClientConfig, RecentMessage, Logger } from "./types.js";
import type { OffloadApiClient } from "./offload-api-client.js";
import { estimateAllTokens, estimateMessageTokens } from "./token-estimator.js";
const DEFAULT_CONTEXT_WINDOW = 128000;
/** compact target: keep messages until total <= contextWindow * TARGET_RATIO */
const COMPACT_TARGET_RATIO = 0.5;
/** When truncating a single large tool_result, keep at most this many chars */
const TOOL_RESULT_TRUNCATE_CHARS = 2000;
// ─── Message role helpers (handle multiple formats) ─────────────────────────
function getMsgRole(msg: any): string {
return msg?.role ?? msg?.message?.role ?? msg?.type ?? "";
}
function isToolResult(msg: any): boolean {
const role = getMsgRole(msg);
if (role === "tool" || role === "toolResult" || role === "tool_result") return true;
// Anthropic: user message with tool_result content blocks
if (role === "user" && Array.isArray(msg?.content)) {
return msg.content.some((b: any) => b?.type === "tool_result");
}
return false;
}
function isAssistantWithToolUse(msg: any): boolean {
const role = getMsgRole(msg);
if (role !== "assistant") return false;
const content = msg?.type === "message" ? msg?.message?.content : msg?.content;
if (!Array.isArray(content)) return false;
return content.some((b: any) => b?.type === "tool_use" || b?.type === "toolCall");
}
/**
* Truncate tool_result content in-place, returning a shallow-cloned message.
*/
function truncateToolResult(msg: any, maxChars: number): any {
const clone = JSON.parse(JSON.stringify(msg));
const content = clone.type === "message" ? clone.message?.content : clone.content;
if (typeof content === "string" && content.length > maxChars) {
const truncated = content.slice(0, maxChars) + "\n...[truncated]";
if (clone.type === "message") clone.message.content = truncated;
else clone.content = truncated;
return clone;
}
if (Array.isArray(content)) {
for (const block of content) {
if (typeof block?.text === "string" && block.text.length > maxChars) {
block.text = block.text.slice(0, maxChars) + "\n...[truncated]";
}
if (typeof block?.content === "string" && block.content.length > maxChars) {
block.content = block.content.slice(0, maxChars) + "\n...[truncated]";
}
}
return clone;
}
return clone;
}
export class OffloadContextEngine {
/** Per-session state cache. */
private sessions = new Map<string, {
lastAccessMs: number; // ← NEW: track last access time for cleanup
lastKnownTotalTokens: number;
lastKnownMsgCount: number;
lastL15PromptHash: string;
cachedPrompt?: string;
cachedRecentMessages: RecentMessage[];
cachedRecentContext?: string;
}>();
/** Get or create per-session state. */
private getSession(sessionKey: string) {
let s = this.sessions.get(sessionKey);
if (!s) {
s = {
lastAccessMs: Date.now(), // ← NEW
lastKnownTotalTokens: 0,
lastKnownMsgCount: 0,
lastL15PromptHash: "",
cachedRecentMessages: [],
};
this.sessions.set(sessionKey, s);
} else {
s.lastAccessMs = Date.now(); // ← NEW: update on access
}
return s;
}
/**
* Reset session state (call when session is /new'd or destroyed).
*/
resetSession(sessionKey: string): void {
this.sessions.delete(sessionKey);
}
/**
* Clear all session states (emergency shutdown).
*/
clearAllSessions(): void {
const n = this.sessions.size;
this.sessions.clear();
if (n > 0) {
this.logger.info(`[offload-client] cleared ${n} session states`);
}
}
/**
* Get the cached context for a session (for after_tool_call hook to send with ingest).
*/
getContext(sessionKey?: string): { prompt?: string; recentMessages?: RecentMessage[] } | undefined {
if (!sessionKey) return undefined;
const s = this.sessions.get(sessionKey);
if (!s || (!s.cachedPrompt && s.cachedRecentMessages.length === 0)) return undefined;
return { prompt: s.cachedPrompt, recentMessages: s.cachedRecentMessages };
}
/**
* Get the cached formatted context string for a session (for legacy compatibility).
*/
getRecentContext(sessionKey?: string): string | undefined {
if (!sessionKey) return undefined;
return this.sessions.get(sessionKey)?.cachedRecentContext;
}
constructor(
private client: OffloadApiClient,
private config: OffloadClientConfig,
private logger: Logger,
) {}
get info() {
return {
id: "memory-tencentdb",
name: "Offload Client Context Engine",
version: "2.0.0",
ownsCompaction: true,
};
}
/**
* bootstrap called when a new session starts (e.g. /new command).
* Resets per-session cached state.
*/
async bootstrap(params: { sessionKey?: string; sessionId?: string }) {
const sk = params.sessionKey ?? params.sessionId;
if (sk) this.resetSession(sk);
return { bootstrapped: true };
}
/**
* ingest no-op for client mode (ingest is handled by after_tool_call hook).
* Required by the framework ContextEngine interface.
*/
async ingest(_params: {
sessionId: string;
sessionKey?: string;
message: any;
isHeartbeat?: boolean;
}) {
return { ingested: true };
}
/**
* compact record the framework's authoritative token count for calibration,
* then defer actual compaction to assemble().
*/
async compact(params: {
sessionId: string;
sessionKey?: string;
sessionFile: string;
tokenBudget?: number;
force?: boolean;
currentTokenCount?: number;
compactionTarget?: "budget" | "threshold";
customInstructions?: string;
runtimeContext?: any;
}) {
// Record framework token count for calibration
if (params.currentTokenCount && params.currentTokenCount > 0) {
const sk = params.sessionKey ?? params.sessionId;
const s = this.getSession(sk);
s.lastKnownTotalTokens = params.currentTokenCount;
this.logger.debug?.(
`[offload-client] compact: calibration updated, knownTokens=${s.lastKnownTotalTokens}`,
);
}
const contextWindow = params.tokenBudget ?? DEFAULT_CONTEXT_WINDOW;
const targetTokens = Math.floor(contextWindow * COMPACT_TARGET_RATIO);
this.logger.info(
`[offload-client] compact: sessionKey=${params.sessionKey ?? params.sessionId}, ` +
`budget=${contextWindow}, target=${targetTokens}, ` +
`currentTokens=${params.currentTokenCount}, force=${params.force}`,
);
return {
ok: true,
compacted: false,
reason: "offload-client: compaction deferred to assemble()",
};
}
/**
* Resolve the best known token total for calibration.
* Uses lastKnownTotalTokens only if message count hasn't changed significantly.
*/
private resolveCalibrationTokens(sessionKey: string, msgCount: number): number | undefined {
const s = this.sessions.get(sessionKey);
if (!s || s.lastKnownTotalTokens <= 0) return undefined;
// If message count changed by >20%, the cached total is stale — skip calibration
if (s.lastKnownMsgCount > 0 && Math.abs(msgCount - s.lastKnownMsgCount) / s.lastKnownMsgCount > 0.2) {
return undefined;
}
return s.lastKnownTotalTokens;
}
/**
* Local brute-force compaction: keep tail messages up to target budget,
* respecting tool pairs and truncating oversized tool_results.
* Used as fallback when server compaction is unavailable.
*/
private localCompact(messages: any[], contextWindow: number, sessionKey: string): any[] {
const targetTokens = Math.floor(contextWindow * COMPACT_TARGET_RATIO);
const knownTokens = this.resolveCalibrationTokens(sessionKey, messages.length);
const { perMessage } = estimateAllTokens(messages, knownTokens);
const n = messages.length;
// Step 1: scan from tail, find the cut index
let cumTokens = 0;
let cutIdx = n; // everything before cutIdx gets deleted
for (let i = n - 1; i >= 0; i--) {
cumTokens += perMessage[i];
if (cumTokens > targetTokens) {
cutIdx = i + 1;
break;
}
cutIdx = i;
}
// Never delete the very first user message (index 0)
if (cutIdx <= 0) cutIdx = 0;
// Step 2: expand cut boundary to respect tool pairs
// If cutIdx lands inside a tool pair, move it to include the full pair.
// 2a: If msg at cutIdx is a tool_result, its paired assistant+tool_use is
// before cutIdx (would be deleted). Move cutIdx back to include the pair.
while (cutIdx < n && isToolResult(messages[cutIdx])) {
cutIdx++;
}
// 2b: If msg at cutIdx-1 (last deleted) is assistant+tool_use, its tool_result
// at cutIdx would be orphaned. Pull cutIdx back to keep the pair.
while (cutIdx > 0 && cutIdx < n && isAssistantWithToolUse(messages[cutIdx - 1])) {
cutIdx--;
}
// Step 3: build retained array
const retained = messages.slice(cutIdx);
if (retained.length === 0) {
return [...messages]; // safety: don't delete everything
}
const deletedCount = cutIdx;
let retainedTokens = 0;
for (let i = cutIdx; i < n; i++) retainedTokens += perMessage[i];
this.logger.info(
`[offload-client] localCompact: deleted ${deletedCount}/${n} msgs, ` +
`retained ${retained.length} msgs, tokens=${retainedTokens}/${targetTokens} target`,
);
// Step 4: if still over target and there's a large tool_result, truncate it
if (retainedTokens > targetTokens) {
let maxTrIdx = -1;
let maxTrTokens = 0;
for (let i = 0; i < retained.length; i++) {
if (isToolResult(retained[i])) {
const t = estimateMessageTokens(retained[i]);
if (t > maxTrTokens) {
maxTrTokens = t;
maxTrIdx = i;
}
}
}
if (maxTrIdx >= 0 && maxTrTokens > TOOL_RESULT_TRUNCATE_CHARS / 4) {
retained[maxTrIdx] = truncateToolResult(retained[maxTrIdx], TOOL_RESULT_TRUNCATE_CHARS);
const newTokens = estimateMessageTokens(retained[maxTrIdx]);
retainedTokens = retainedTokens - maxTrTokens + newTokens;
this.logger.info(
`[offload-client] localCompact: truncated tool_result[${maxTrIdx}] ` +
`${maxTrTokens}${newTokens} tokens, total now=${retainedTokens}`,
);
}
}
return retained;
}
// ─── L1.5 Trigger ───────────────────────────────────────────────────────────
/**
* Trigger L1.5 task judgment via ingest API when a new user prompt is detected.
* Fire-and-forget does not block the assemble flow.
*/
private triggerL15IfNeeded(prompt: string | undefined, messages: any[], sessionKey: string): void {
if (!prompt || typeof prompt !== "string" || prompt.length === 0) return;
// Skip system/internal prompts that are not user-initiated
if (this.isInternalPrompt(prompt)) {
this.logger.debug?.(`[offload-client] L1.5 skipped: internal prompt (${prompt.slice(0, 60)})`);
return;
}
// Always update cached context for after_tool_call hook (L1 needs it)
const recentMsgs = this.buildRecentMessages(prompt, messages);
const s = this.getSession(sessionKey);
s.cachedPrompt = prompt.slice(0, 500);
s.cachedRecentMessages = recentMsgs;
s.cachedRecentContext = this.formatContextForL1(prompt, recentMsgs);
// Dedup: skip L1.5 if same prompt as last trigger for this session
const hash = this.simpleHash(prompt);
if (s.lastL15PromptHash === hash) {
this.logger.debug?.(`[offload-client] L1.5 skipped: same prompt hash (${hash})`);
return;
}
s.lastL15PromptHash = hash;
this.logger.info(
`[offload-client] L1.5 triggered: promptHash=${hash}, recentMsgs=${recentMsgs.length}`,
);
// Fire-and-forget L1.5
this.client.ingestL15(sessionKey, prompt.slice(0, 500), recentMsgs).catch((err) => {
this.logger.warn(`[offload-client] L1.5 ingestL15 failed: ${err}`);
});
}
/**
* Detect internal/system prompts that should not trigger L1.5.
* These are framework-generated messages, not user-initiated conversations.
*/
private isInternalPrompt(prompt: string): boolean {
// Compaction flush prompts
if (prompt.startsWith("Pre-compaction")) return true;
// Inter-session routing messages
if (prompt.startsWith("[Inter-session message]")) return true;
// Heartbeat/keepalive
if (prompt.includes("HEARTBEAT") || prompt.includes("heartbeat")) return true;
return false;
}
/**
* Build structured RecentMessage[] for ingest API.
* Filters: user/assistant text only, no tool calls, no heartbeats.
* Max 5 recent turns, 400 chars per message.
*/
private buildRecentMessages(prompt: string, messages: any[]): RecentMessage[] {
const normalizedPrompt = prompt.trim().slice(0, 200).toLowerCase();
// Scan messages, collect user/assistant pairs
const pairs: RecentMessage[] = [];
for (const msg of messages) {
const role = getMsgRole(msg);
// Skip tool messages entirely
if (isToolResult(msg) || isAssistantWithToolUse(msg)) continue;
if (role === "tool" || role === "toolResult" || role === "tool_result") continue;
if (role === "user") {
const text = this.extractMsgText(msg);
if (!text || text.length <= 5) continue;
if (text.includes("HEARTBEAT") || text.includes("heartbeat")) continue;
const trimmed = text.slice(0, 400);
// Skip if it matches current prompt
const normalizedText = trimmed.slice(0, 200).toLowerCase();
if (normalizedPrompt && (normalizedText === normalizedPrompt || normalizedText.startsWith(normalizedPrompt) || normalizedPrompt.startsWith(normalizedText))) continue;
pairs.push({ role: "user", content: trimmed });
} else if (role === "assistant") {
const text = this.extractMsgText(msg);
if (!text || text.length <= 10) continue;
if (text.includes("HEARTBEAT") || text.includes("heartbeat")) continue;
pairs.push({ role: "assistant", content: text.slice(0, 400) });
}
}
// Keep last N messages (max 10 messages ≈ 5 turns)
const recent = pairs.slice(-10);
return recent;
}
/**
* Format context string for L1 executor (recent-context.txt).
*/
private formatContextForL1(prompt: string, recentMsgs: RecentMessage[]): string {
const parts: string[] = [];
if (recentMsgs.length > 0) {
parts.push("历史消息,可作为参考:");
for (const m of recentMsgs) {
parts.push(`[${m.role === "user" ? "User" : "Assistant"}]: ${m.content}`);
}
}
parts.push(`\n最新user message\n[User]: ${prompt.slice(0, 500)}`);
return parts.join("\n");
}
/**
* Extract text content from a message.
*/
private extractMsgText(msg: any): string {
const content = msg?.content ?? msg?.message?.content ?? "";
if (typeof content === "string") return content;
if (Array.isArray(content)) {
return content
.map((b: any) => (typeof b === "string" ? b : b?.text ?? ""))
.join("");
}
return "";
}
/**
* Simple string hash for prompt deduplication.
*/
private simpleHash(str: string): string {
let hash = 0;
for (let i = 0; i < str.length; i++) {
hash = ((hash << 5) - hash + str.charCodeAt(i)) | 0;
}
return hash.toString(36);
}
/**
* assemble estimate ratio call server compaction fallback to localCompact.
* Framework calls this to build the model context for each turn.
*/
async assemble(params: {
sessionId: string;
sessionKey?: string;
messages?: any[];
tokenBudget?: number;
prompt?: string;
availableTools?: Set<string>;
citationsMode?: string;
model?: string;
}) {
const { messages, sessionKey, sessionId } = params;
if (!messages || messages.length === 0) {
return { messages: messages ? [...messages] : [], estimatedTokens: 0 };
}
const sk = sessionKey ?? sessionId ?? "unknown";
// ── L1.5 trigger: fire-and-forget on new user prompt ──
this.triggerL15IfNeeded(params.prompt, messages, sk);
const contextWindow = params.tokenBudget ?? DEFAULT_CONTEXT_WINDOW;
// Don't use framework's knownTokens for calibration — our tiktoken is already precise.
// Framework's currentTokenCount may use a different calculation method (e.g. chars/4).
const { total, perMessage } = estimateAllTokens(messages);
const ratio = total / contextWindow;
// Update calibration state for future calls
const s = this.getSession(sk);
s.lastKnownTotalTokens = total;
s.lastKnownMsgCount = messages.length;
// Below client threshold — skip compaction
if (ratio < this.config.compactionRatio) {
this.logger.debug?.(
`[offload-client] assemble: ratio=${(ratio * 100).toFixed(1)}% < ${(this.config.compactionRatio * 100).toFixed(0)}%, skip`,
);
return { messages: [...messages], estimatedTokens: total };
}
this.logger.info(
`[offload-client] assemble: ratio=${(ratio * 100).toFixed(1)}%, msgs=${messages.length}, calling compaction...`,
);
// Try server-side compaction first
const result = await this.client.compaction({
sessionId: sessionKey ?? sessionId ?? "unknown",
messages,
ratio,
contextWindow,
totalTokens: total,
messageTokens: perMessage,
});
if (result) {
const compactedTokens = result.messages.reduce(
(sum: number, msg: any) => sum + estimateMessageTokens(msg),
0,
);
this.logger.info(
`[offload-client] server compaction done: level=${result.report.resolvedLevel}, ` +
`${result.report.originalCount}${result.report.compactedCount} msgs, ` +
`mild=${result.report.mildReplacements}, agg=${result.report.aggressiveDeleted}, ` +
`em=${result.report.emergencyDeleted}, mmd=${result.report.mmdInjected}`,
);
return { messages: result.messages, estimatedTokens: compactedTokens };
}
// Fallback: local brute-force compaction
this.logger.warn("[offload-client] server compaction failed, falling back to local compact");
const compacted = this.localCompact(messages, contextWindow, sk);
const compactedTokens = compacted.reduce(
(sum: number, msg: any) => sum + estimateMessageTokens(msg),
0,
);
return { messages: compacted, estimatedTokens: compactedTokens };
}
/**
* afterTurn no-op (ingest is handled by after_tool_call hook).
*/
async afterTurn() {}
}
@@ -0,0 +1,53 @@
/**
* offload-client after_tool_call hook handler.
* Fire-and-forget: sends tool pair + context to ingest API for L1 processing.
*/
import type { OffloadApiClient } from "../offload-api-client.js";
import type { OffloadClientConfig, ToolPairPayload, RecentMessage, Logger } from "../types.js";
export interface AfterToolCallEvent {
toolName: string;
toolCallId: string;
params?: unknown;
result?: unknown;
error?: string;
durationMs?: number;
}
/**
* Create the after_tool_call hook handler.
* Sends each tool call result to the server for L1 processing.
*
* @param getContext Optional getter for { prompt, recentMessages } context per session.
*/
export function createAfterToolCallHandler(
client: OffloadApiClient,
config: OffloadClientConfig,
logger: Logger,
getContext?: (sessionKey: string) => { prompt?: string; recentMessages?: RecentMessage[] } | undefined,
) {
return (event: AfterToolCallEvent, ctx: { sessionKey?: string; sessionId?: string }) => {
const sessionId = ctx.sessionKey ?? ctx.sessionId;
logger.debug?.(
`[offload-client] after_tool_call: tool=${event.toolName}, session=${sessionId ?? "(none)"}, callId=${event.toolCallId ?? "(none)"}`,
);
if (!sessionId) return;
const toolPair: ToolPairPayload = {
toolName: event.toolName,
toolCallId: event.toolCallId,
params: event.params ?? {},
result: event.result,
error: event.error,
timestamp: new Date().toISOString(),
durationMs: event.durationMs,
};
const context = getContext?.(sessionId);
// Fire-and-forget — do not block the LLM flow
client.ingestWithContext(sessionId, [toolPair], context?.prompt, context?.recentMessages).catch((err) => {
logger.warn(`[offload-client] ingest fire-and-forget error: ${err}`);
});
};
}
+101
View File
@@ -0,0 +1,101 @@
/**
* offload-client Plugin registration entry point.
* Stateless, server-delegated offload client.
* 1 hook (after_tool_call) + 1 Context Engine (assemble compaction API).
*/
import type { OffloadClientConfig, Logger } from "./types.js";
import { defaultOffloadClientConfig } from "./types.js";
import { OffloadApiClient } from "./offload-api-client.js";
import { OffloadContextEngine } from "./context-engine.js";
import { createAfterToolCallHandler } from "./hooks/after-tool-call.js";
export interface OpenClawPluginApi {
on: (hookName: string, handler: (...args: any[]) => any) => void;
registerContextEngine: (id: string, factoryOrInstance: any) => any;
logger?: Logger;
}
/**
* Register the offload-client plugin.
* Call this from the main plugin's register() when offload-client config is enabled.
*/
export function registerOffloadClient(api: OpenClawPluginApi, userConfig: Partial<OffloadClientConfig>): void {
const config: OffloadClientConfig = { ...defaultOffloadClientConfig(), ...userConfig };
const logger: Logger = api.logger ?? { info: console.log, warn: console.warn, error: console.error, debug: console.debug };
if (!config.enabled) {
logger.info("[offload-client] disabled by config");
return;
}
if (!config.serverUrl || !config.apiKey || !config.serviceId) {
logger.error("[offload-client] missing required config: serverUrl, apiKey, or serviceId");
return;
}
const client = new OffloadApiClient(config, logger);
// Context Engine: occupies slot, assemble() calls compaction API
const engine = new OffloadContextEngine(client, config, logger);
// Hook: fire-and-forget ingest on every tool call (with context from engine per session)
const afterToolCallHandler = createAfterToolCallHandler(
client, config, logger,
(sessionKey) => engine.getContext(sessionKey),
config.agentName, // ← NEW: pass agentName for sessionId construction
);
api.on("after_tool_call", afterToolCallHandler);
// ── Memory management hooks ──
// agent_end: clear token cache after each agent turn
api.on("agent_end", (_event: any, ctx: { sessionKey?: string; sessionId?: string }) => {
const sk = ctx.sessionKey ?? ctx.sessionId;
if (sk) {
engine.resetSession(sk);
logger.debug?.(`[offload-client] reset session state: ${sk}`);
}
});
// gateway_stop: emergency cleanup on shutdown
api.on("gateway_stop", async () => {
engine.clearAllSessions();
logger.info("[offload-client] all session states cleared on gateway_stop");
});
try {
const result = api.registerContextEngine("memory-tencentdb", () => engine) as any;
if (result?.ok === false) {
logger.error(
`[offload-client] Context Engine slot occupied by "${result.existingOwner ?? "unknown"}". ` +
`Compaction disabled — only ingest will work.`,
);
} else {
logger.info("[offload-client] Context Engine registered");
}
} catch (err) {
logger.error(`[offload-client] registerContextEngine failed: ${err}. Compaction disabled.`);
}
// ── Health check (async, non-blocking) ──
client.checkHealth().then((ok) => {
if (!ok) {
logger.warn(
`[offload-client] ⚠️ Server ${config.serverUrl} unreachable! ` +
`Ingest calls will fail silently until server becomes available.`
);
} else {
logger.info(`[offload-client] Server health OK: ${config.serverUrl}`);
}
}).catch((_err) => {
// ignore
});
logger.info(`[offload-client] registered (server=${config.serverUrl})`);
}
export { OffloadApiClient } from "./offload-api-client.js";
export { OffloadContextEngine } from "./context-engine.js";
export { createAfterToolCallHandler } from "./hooks/after-tool-call.js";
export { estimateTokens, estimateMessageTokens, estimateAllTokens } from "./token-estimator.js";
export type { OffloadClientConfig, ToolPairPayload, CompactionResult, CompactionReport, Logger } from "./types.js";
+177
View File
@@ -0,0 +1,177 @@
/**
* offload-client HTTP client for Offload Server v2 API.
*/
import type { OffloadClientConfig, ToolPairPayload, RecentMessage, CompactionResult, Logger } from "./types.js";
export class OffloadApiClient {
constructor(
private config: OffloadClientConfig,
private logger: Logger,
) {}
/** Health check: GET /v2/offload/health. Returns true if server is reachable (any HTTP response = reachable). */
async checkHealth(): Promise<boolean> {
try {
const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), 5000);
const res = await fetch(`${this.config.serverUrl}/v2/offload/health`, {
method: "GET",
headers: { Authorization: `Bearer ${this.config.apiKey}` },
signal: controller.signal,
});
clearTimeout(timeoutId);
// Any HTTP response (including 401/403) means server is reachable
return res.status < 500;
} catch {
return false;
}
}
/**
* Fire-and-forget: send tool pairs to ingest endpoint.
* Does not throw failures are logged as warnings.
*/
async ingest(sessionId: string, toolPairs: ToolPairPayload[]): Promise<void> {
return this.ingestWithContext(sessionId, toolPairs, undefined, undefined);
}
/**
* Fire-and-forget: send tool pairs + optional context to ingest endpoint.
* When prompt/recentMessages are provided, server triggers L1 with context (skip L1.5).
*/
async ingestWithContext(
sessionId: string,
toolPairs: ToolPairPayload[],
prompt?: string,
recentMessages?: RecentMessage[],
): Promise<void> {
const url = `${this.config.serverUrl}/v2/offload/ingest`;
const payload: Record<string, unknown> = {
session_id: sessionId,
tool_pairs: toolPairs.map((tp) => ({
tool_name: tp.toolName,
tool_call_id: tp.toolCallId,
params: tp.params,
result: tp.result,
error: tp.error,
timestamp: tp.timestamp,
duration_ms: tp.durationMs,
})),
};
if (prompt) payload.prompt = prompt;
if (recentMessages && recentMessages.length > 0) payload.recent_messages = recentMessages;
const body = JSON.stringify(payload);
try {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), this.config.ingestTimeoutMs);
await fetch(url, {
method: "POST",
headers: this.buildHeaders(),
body,
signal: controller.signal,
});
clearTimeout(timer);
} catch (err) {
this.logger.warn(`[offload-client] ingest failed: ${err}`);
}
}
/**
* Fire-and-forget: trigger L1.5 task judgment via ingest endpoint.
* Sends prompt + recentMessages (empty toolPairs) to activate the L1.5 path on the server.
*/
async ingestL15(sessionId: string, prompt: string, recentMessages?: RecentMessage[]): Promise<void> {
const url = `${this.config.serverUrl}/v2/offload/ingest`;
const payload: Record<string, unknown> = {
session_id: sessionId,
tool_pairs: [],
prompt,
};
if (recentMessages && recentMessages.length > 0) payload.recent_messages = recentMessages;
const body = JSON.stringify(payload);
try {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), this.config.ingestTimeoutMs);
const response = await fetch(url, {
method: "POST",
headers: this.buildHeaders(),
body,
signal: controller.signal,
});
clearTimeout(timer);
if (!response.ok) {
this.logger.warn(`[offload-client] ingestL15 returned ${response.status}`);
}
} catch (err) {
this.logger.warn(`[offload-client] ingestL15 failed: ${err}`);
}
}
/**
* Synchronous compaction call. Returns compressed messages + report.
* Returns null on timeout/failure (caller should keep original messages).
*/
async compaction(req: {
sessionId: string;
messages: any[];
ratio: number;
contextWindow: number;
totalTokens: number;
messageTokens?: number[];
}): Promise<CompactionResult | null> {
const url = `${this.config.serverUrl}/v2/offload/compact`;
const body = JSON.stringify({
session_id: req.sessionId,
messages: req.messages,
ratio: req.ratio,
context_window: req.contextWindow,
total_tokens: req.totalTokens,
message_tokens: req.messageTokens,
});
try {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), this.config.compactionTimeoutMs);
const response = await fetch(url, {
method: "POST",
headers: this.buildHeaders(),
body,
signal: controller.signal,
});
clearTimeout(timer);
if (!response.ok) {
this.logger.warn(`[offload-client] compaction returned ${response.status}`);
return null;
}
const json = (await response.json()) as any;
if (json.code !== 0 || !json.data) {
this.logger.warn(`[offload-client] compaction error: ${json.message ?? "unknown"}`);
return null;
}
return { messages: json.data.messages, report: json.data.report };
} catch (err) {
this.logger.warn(`[offload-client] compaction failed: ${err}`);
return null;
}
}
private buildHeaders(): Record<string, string> {
return {
"Content-Type": "application/json",
Authorization: `Bearer ${this.config.apiKey}`,
"X-TDAI-Service-Id": this.config.serviceId,
};
}
}
+227
View File
@@ -0,0 +1,227 @@
/**
* offload-client Token estimation using tiktoken (precise).
* Aligned with server-side preciseMessageTokens: only counts LLM-visible content.
*
* Strategy:
* - Primary: tiktoken BPE encoding (o200k_base) on role + content
* - Fallback: CJK-aware heuristic if tiktoken fails
*
* `estimateAllTokens` still supports optional calibration from framework-reported
* totalTokens, but with tiktoken the drift should be minimal.
*/
import { getEncoding, type Tiktoken } from "js-tiktoken";
// ─── Tiktoken Encoder (lazy singleton) ──────────────────────────────────────
let _encoder: Tiktoken | null = null;
function getEncoder(): Tiktoken {
if (!_encoder) _encoder = getEncoding("o200k_base");
return _encoder;
}
// ─── Calibration constants ──────────────────────────────────────────────────
/** If heuristic drifts > 15% from known total, apply linear scaling. */
const CALIBRATION_THRESHOLD = 0.15;
/** Clamp calibration factor to prevent extreme scaling from noisy estimates. */
const CALIBRATION_FACTOR_MIN = 0.5;
const CALIBRATION_FACTOR_MAX = 3.0;
// ─── LLM-visible text extraction (must match server-side extractLlmVisibleText) ──
/**
* Extract the LLM-visible portion of a message (role + content only).
* Matches server-side preciseMessageTokens logic exactly.
*/
function extractLlmVisibleText(msg: any): string {
const role: string = msg?.role ?? msg?.message?.role ?? "";
const rawContent = msg?.content ?? msg?.message?.content ?? "";
let contentStr: string;
if (typeof rawContent === "string") {
contentStr = rawContent;
} else if (Array.isArray(rawContent)) {
const parts: string[] = [];
for (const block of rawContent) {
if (typeof block === "string") {
parts.push(block);
} else if (block?.type === "text" && typeof block.text === "string") {
parts.push(block.text);
} else if (block?.type === "tool_use" || block?.type === "toolCall") {
parts.push(block.name ?? block.toolName ?? "");
if (block.arguments) {
parts.push(typeof block.arguments === "string" ? block.arguments : JSON.stringify(block.arguments));
}
if (block.input) {
parts.push(typeof block.input === "string" ? block.input : JSON.stringify(block.input));
}
} else if (block?.type === "tool_result") {
if (typeof block.content === "string") parts.push(block.content);
else if (block.content) parts.push(JSON.stringify(block.content));
} else {
parts.push(JSON.stringify(block));
}
}
contentStr = parts.join("\n");
} else {
contentStr = JSON.stringify(rawContent);
}
return `${role}\n${contentStr}`;
}
// ─── Per-message token cache (WeakMap — auto GC when msg object is released) ──
const _tokenCache = new WeakMap<object, number>();
/** Max messages to tiktoken precisely per call. Beyond this, old messages use heuristic. */
const PRECISE_BUDGET = 200;
// ─── Core estimation ────────────────────────────────────────────────────────
/**
* Estimate tokens for a text string using tiktoken.
* Falls back to CJK-aware heuristic on error.
*/
export function estimateTokens(text: string): number {
if (!text) return 0;
try {
return getEncoder().encode(text).length;
} catch {
return heuristicTokens(text);
}
}
/**
* Estimate tokens for a single message using tiktoken (with WeakMap cache).
* Counts only LLM-visible content (role + content), aligned with server.
*/
export function estimateMessageTokens(msg: any): number {
if (msg == null) return 0;
// Cache hit
if (typeof msg === "object" && _tokenCache.has(msg)) {
return _tokenCache.get(msg)!;
}
const tokens = _computeMessageTokens(msg);
// Cache store (only for objects)
if (typeof msg === "object" && msg !== null) {
_tokenCache.set(msg, tokens);
}
return tokens;
}
function _computeMessageTokens(msg: any): number {
try {
const text = extractLlmVisibleText(msg);
return getEncoder().encode(text).length + 4; // +4 for message framing overhead
} catch {
return heuristicTokens(extractLlmVisibleText(msg));
}
}
/**
* Fast heuristic estimate for a message (no tiktoken, ~0.01ms per msg).
* Used for old messages when total count exceeds PRECISE_BUDGET.
*/
function heuristicMessageTokens(msg: any): number {
if (msg == null) return 0;
return heuristicTokens(extractLlmVisibleText(msg));
}
// ─── Calibrated batch estimation ────────────────────────────────────────────
/**
* Estimate total tokens and per-message tokens for a message array.
*
* Performance strategy:
* - If messages.length <= PRECISE_BUDGET (200): tiktoken all (with cache)
* - If messages.length > PRECISE_BUDGET: heuristic for old, tiktoken for recent N
* Then calibrate old estimates using recent tiktoken as reference.
*
* @param knownTotalTokens Optional authoritative total from the framework.
*/
export function estimateAllTokens(
messages: any[],
knownTotalTokens?: number,
): { total: number; perMessage: number[] } {
const n = messages.length;
if (n === 0) return { total: 0, perMessage: [] };
let raw: number[];
if (n <= PRECISE_BUDGET) {
// Small batch: tiktoken all (cache makes repeat calls fast)
raw = messages.map((msg) => estimateMessageTokens(msg));
} else {
// Large batch: tiktoken recent, heuristic old, calibrate
const preciseStart = n - PRECISE_BUDGET;
raw = new Array(n);
// Recent messages: precise (also populates cache for next call)
let preciseSum = 0;
let heuristicSumForRecent = 0;
for (let i = preciseStart; i < n; i++) {
raw[i] = estimateMessageTokens(messages[i]);
preciseSum += raw[i];
heuristicSumForRecent += heuristicMessageTokens(messages[i]);
}
// Calibration factor from recent messages
const calibFactor = heuristicSumForRecent > 0 ? preciseSum / heuristicSumForRecent : 1;
// Old messages: heuristic × calibration factor (or cache hit if available)
for (let i = 0; i < preciseStart; i++) {
if (typeof messages[i] === "object" && _tokenCache.has(messages[i])) {
raw[i] = _tokenCache.get(messages[i])!;
} else {
raw[i] = Math.max(1, Math.round(heuristicMessageTokens(messages[i]) * calibFactor));
}
}
}
let rawTotal = raw.reduce((s, v) => s + v, 0);
// External calibration (from framework totalTokens)
if (!knownTotalTokens || knownTotalTokens <= 0 || rawTotal <= 0) {
return { total: rawTotal, perMessage: raw };
}
const drift = Math.abs(knownTotalTokens - rawTotal) / knownTotalTokens;
if (drift <= CALIBRATION_THRESHOLD) {
return { total: rawTotal, perMessage: raw };
}
// Apply linear calibration factor, clamped
const factor = Math.max(
CALIBRATION_FACTOR_MIN,
Math.min(CALIBRATION_FACTOR_MAX, knownTotalTokens / rawTotal),
);
const calibrated = raw.map((v) => Math.max(1, Math.round(v * factor)));
const calibratedTotal = calibrated.reduce((s, v) => s + v, 0);
return { total: calibratedTotal, perMessage: calibrated };
}
// ─── Heuristic fallback ─────────────────────────────────────────────────────
function countCjkChars(text: string): number {
let n = 0;
for (const ch of text) {
const c = ch.codePointAt(0)!;
if (
(c >= 0x4e00 && c <= 0x9fff) ||
(c >= 0x3400 && c <= 0x4dbf) ||
(c >= 0xf900 && c <= 0xfaff)
) {
n++;
}
}
return n;
}
function heuristicTokens(text: string): number {
if (!text) return 0;
const cjk = countCjkChars(text);
const rest = Math.max(0, text.length - cjk);
return Math.max(1, Math.ceil(cjk / 1.7 + rest / 4));
}
+80
View File
@@ -0,0 +1,80 @@
/**
* offload-client Type definitions.
*/
// ─── Plugin Configuration ────────────────────────────────────────────────────
export interface OffloadClientConfig {
enabled: boolean;
/** Offload server base URL (e.g. "http://localhost:9100"). */
serverUrl: string;
/** Bearer token for Authorization header. */
apiKey: string;
/** X-TDAI-Service-Id header value. */
serviceId: string;
/** Agent name for sessionId construction. Default "default". */
agentName?: string;
/** Client-side threshold: skip compaction request when ratio < this value. Default 0.5. */
compactionRatio: number;
/** Ingest request timeout in ms. Default 5000. */
ingestTimeoutMs: number;
/** Compaction request timeout in ms. Default 30000. */
compactionTimeoutMs: number;
}
export function defaultOffloadClientConfig(): OffloadClientConfig {
return {
enabled: false,
serverUrl: "http://localhost:9100",
apiKey: "",
serviceId: "",
compactionRatio: 0.5,
ingestTimeoutMs: 5000,
compactionTimeoutMs: 30000,
};
}
// ─── Ingest Payload ──────────────────────────────────────────────────────────
export interface ToolPairPayload {
toolName: string;
toolCallId: string;
params: unknown;
result: unknown;
error?: string;
timestamp: string;
durationMs?: number;
}
export interface RecentMessage {
role: "user" | "assistant";
content: string;
}
// ─── Compaction Response ─────────────────────────────────────────────────────
export interface CompactionReport {
resolvedLevel: string;
originalCount: number;
compactedCount: number;
fastPathReplaced: number;
fastPathDeleted: number;
mildReplacements: number;
aggressiveDeleted: number;
emergencyDeleted: number;
mmdInjected: number;
}
export interface CompactionResult {
messages: any[];
report: CompactionReport;
}
// ─── Logger ──────────────────────────────────────────────────────────────────
export interface Logger {
info: (...args: unknown[]) => void;
warn: (...args: unknown[]) => void;
error: (...args: unknown[]) => void;
debug?: (...args: unknown[]) => void;
}
+3
View File
@@ -27,6 +27,8 @@ export interface CallLlmOpts {
timeoutMs?: number;
/** Label for logging (e.g. "L1", "L1.5", "L2") */
label?: string;
/** Instance ID for telemetry metadata */
instanceId?: string;
}
/**
@@ -64,6 +66,7 @@ export async function callLlm(
experimental_telemetry: {
isEnabled: true,
functionId: opts.label ?? "offload-llm",
metadata: { instanceId: opts.instanceId ?? "unknown" },
},
});
+3 -6
View File
@@ -3,6 +3,8 @@
* Ported from context-offload-plugin with updated runtime defaults.
*/
import type { Logger } from "../core/types.js";
// ============================
// Data types
// ============================
@@ -212,12 +214,7 @@ export interface PluginConfig {
// ============================
/** Logger interface used by offload plugin components */
export interface PluginLogger {
info: (msg: string) => void;
warn: (msg: string) => void;
error: (msg: string) => void;
debug?: (msg: string) => void;
}
export type PluginLogger = Logger;
// ============================
// Plugin defaults
@@ -0,0 +1,328 @@
/**
* L3 Compaction Handler orchestrates fast-path, MMD injection, and compression.
* Synchronous API: client awaits the response.
*
* Uses compact-state.json (independent from state.json) to avoid lock contention
* with L1/L1.5/L2 executors that write state.json concurrently.
*/
import type http from "node:http";
import type { StorageAdapter } from "../../core/storage/adapter.js";
import type { OffloadEntry, OffloadState, OffloadExecutorConfig, CompactState } from "../types.js";
import { defaultOffloadState, defaultCompactState } from "../types.js";
import { parseJsonl } from "../parsers/json-utils.js";
import { CompactionRequestSchemaV2 } from "../schemas.js";
import { buildOffloadBasePath } from "../session-utils.js";
import { applyFastPath } from "./fast-path.js";
import { injectActiveMmd, injectHistoryMmds } from "./mmd-injector.js";
import { resolveLevel, mildCompress, aggressiveCompress, emergencyCompress } from "./compressor.js";
import { estimateMessageTokens, extractToolResultId } from "./helpers.js";
import type { Message } from "./helpers.js";
import { traceServerCompaction } from "../opik-tracer.js";
export interface CompactionDeps {
storage: StorageAdapter;
config: OffloadExecutorConfig;
logger: { info: (...args: unknown[]) => void; warn: (...args: unknown[]) => void; error: (...args: unknown[]) => void };
}
export interface CompactionReport {
resolvedLevel: string;
originalCount: number;
compactedCount: number;
fastPathReplaced: number;
fastPathDeleted: number;
mildReplacements: number;
aggressiveDeleted: number;
emergencyDeleted: number;
mmdInjected: number;
}
/**
* Handle POST /v2/offload/compact.
*/
export async function handleCompaction(
req: http.IncomingMessage,
res: http.ServerResponse,
auth: { serviceId: string },
deps: CompactionDeps,
requestId: string,
parseJsonBody: <T>(req: http.IncomingMessage) => Promise<T>,
sendJson: (res: http.ServerResponse, status: number, body: unknown) => void,
successEnvelope: <T>(data: T, requestId: string) => unknown,
errorEnvelope: (code: number, message: string, requestId: string) => unknown,
): Promise<void> {
const body = await parseJsonBody(req);
const parsed = CompactionRequestSchemaV2.safeParse(body);
if (!parsed.success) {
sendJson(res, 400, errorEnvelope(400, parsed.error.message, requestId));
return;
}
const { session_id: sessionId, messages, ratio, context_window: contextWindow, message_tokens: messageTokens } = parsed.data;
let { total_tokens: totalTokens } = parsed.data;
const { storage, config } = deps;
const basePath = buildOffloadBasePath(sessionId);
const originalCount = messages.length;
const compactionStartMs = Date.now();
const { preciseMessageTokens } = await import("./compressor.js");
const originalTotalTokens = totalTokens;
// Compute fixed overhead: system prompt + tool schemas + message framing.
// These are included in clientTotalTokens but NOT in messages.
// overhead = clientTotal - sum(preciseMessageTokens for each message)
const messagesTokenSum = (messages as any[]).reduce(
(s: number, msg: any) => s + preciseMessageTokens(msg), 0,
);
const fixedOverhead = Math.max(0, totalTokens - messagesTokenSum);
deps.logger.info(
`[offload-server] compaction: clientTotal=${totalTokens}, msgsTokens=${messagesTokenSum}, ` +
`overhead=${fixedOverhead}, msgs=${originalCount}, ratio=${ratio.toFixed(2)}`,
);
// Read offload state (L1/L1.5/L2 managed, read-only here) and compact state (L3 owned)
const state = await readOffloadState(storage, basePath);
const compactState = await readCompactState(storage, basePath);
const entriesRaw = await storage.readFile(`${basePath}/entries.jsonl`);
const entries = entriesRaw
? parseJsonl<OffloadEntry>(entriesRaw, (line, err) => {
deps.logger.warn(`[offload-server] compaction: bad JSONL line: ${line}`, err);
})
: [];
// Merge node-mapping.jsonl into entries (L2 writes node_id to a separate file)
const nodeMappingRaw = await storage.readFile(`${basePath}/node-mapping.jsonl`);
if (nodeMappingRaw) {
const mappings = parseJsonl<{ tool_call_id: string; node_id: string }>(nodeMappingRaw);
const nodeMap = new Map(mappings.map((m) => [m.tool_call_id, m.node_id]));
for (const entry of entries) {
if (!entry.node_id && nodeMap.has(entry.tool_call_id)) {
entry.node_id = nodeMap.get(entry.tool_call_id)!;
}
}
}
// Token array: use tiktoken values directly (no calibration against clientTotal).
// clientTotal includes fixedOverhead which is not in messages, so calibration
// would inflate per-message tokens incorrectly.
const tokenArray = buildTokenArray(messages as Message[], messagesTokenSum, messageTokens);
const report: CompactionReport = {
resolvedLevel: "fastpath",
originalCount,
compactedCount: 0,
fastPathReplaced: 0,
fastPathDeleted: 0,
mildReplacements: 0,
aggressiveDeleted: 0,
emergencyDeleted: 0,
mmdInjected: 0,
};
// Step 1: Fast-path re-apply (uses compactState for confirmed/deleted IDs)
const fp = applyFastPath(messages, entries, compactState);
report.fastPathReplaced = fp.replacedCount;
report.fastPathDeleted = fp.deletedCount;
// Recalculate totalTokens after fast-path:
// totalTokens = tiktoken(remaining messages) + fixedOverhead
if (fp.deletedCount > 0) {
const postFpMsgsTokens = (messages as any[]).reduce(
(s: number, msg: any) => s + preciseMessageTokens(msg), 0,
);
totalTokens = postFpMsgsTokens + fixedOverhead;
// Rebuild tokenArray for remaining messages
tokenArray.length = 0;
tokenArray.push(...buildTokenArray(messages as Message[], postFpMsgsTokens, undefined));
}
const effectiveRatio = contextWindow > 0 ? totalTokens / contextWindow : ratio;
// Step 2: Resolve compression level using post-fast-path ratio
const level = resolveLevel(effectiveRatio, {
mildRatio: config.mildOffloadRatio,
aggressiveRatio: config.aggressiveCompressRatio,
emergencyRatio: config.emergencyCompressRatio,
});
report.resolvedLevel = level;
deps.logger.info(
`[offload-server] compaction: level=${level}, ratio=${effectiveRatio.toFixed(2)} (pre-fp=${ratio.toFixed(2)}), msgs=${messages.length}, entries=${entries.length}`,
);
// Step 3: Mild compression
if (level === "mild" || level === "aggressive" || level === "emergency") {
const mild = mildCompress(messages, entries);
report.mildReplacements = mild.replacedCount;
compactState.confirmedOffloadIds.push(...mild.confirmedIds);
// Sync tokenArray after mild replacements (content changed)
// Incremental update: only recalculate tokens for messages that were actually replaced
if (mild.replacedCount > 0) {
const confirmedSet = new Set(mild.confirmedIds);
let tokenDelta = 0;
for (let i = 0; i < messages.length; i++) {
const msg = messages[i] as any;
// Use extractToolResultId which handles all formats (OpenAI, Anthropic content blocks)
const tid = extractToolResultId(msg);
if (!tid || !confirmedSet.has(tid)) continue;
const newTokens = preciseMessageTokens(msg);
tokenDelta += newTokens - tokenArray[i];
tokenArray[i] = newTokens;
}
totalTokens += tokenDelta;
}
}
// Step 4: Aggressive compression
// Target: just below the aggressive trigger threshold (leave ~5% headroom)
let aggRemainingTokens = totalTokens; // track for emergency
if (level === "aggressive" || level === "emergency") {
const aggTargetTokens = Math.floor(contextWindow * (config.aggressiveCompressRatio - 0.05));
// Skip aggressive if mild already brought tokens below target
if (totalTokens <= aggTargetTokens) {
deps.logger.info(
`[offload-server] aggressive skipped: mild already reduced tokens to ${totalTokens} (target=${aggTargetTokens})`,
);
} else {
const agg = aggressiveCompress(messages, aggTargetTokens, tokenArray, totalTokens);
report.aggressiveDeleted = agg.deletedCount;
aggRemainingTokens = agg.remainingTokens;
compactState.deletedOffloadIds.push(...agg.deletedIds);
// Inject history MMDs for deleted entries
if (agg.deletedIds.length > 0) {
const mmdBudget = Math.floor(contextWindow * 0.1 / 4); // 10% of context, in estimated tokens
const hist = await injectHistoryMmds(
messages, agg.deletedIds, entries, state, storage, basePath, mmdBudget,
);
report.mmdInjected += hist.injectedCount;
// Sync tokenArray for any injected MMD messages (calibrate against messages-only tokens)
if (hist.injectedCount > 0) {
const postAggMsgsTokens = (messages as any[]).reduce(
(s: number, msg: any) => s + preciseMessageTokens(msg), 0,
);
tokenArray.length = 0;
tokenArray.push(...buildTokenArray(messages as Message[], postAggMsgsTokens, undefined));
}
}
}
}
// Step 6: Emergency compression
// Target: just below the aggressive threshold (so next turn won't immediately re-trigger)
if (level === "emergency") {
const emTargetTokens = Math.floor(contextWindow * (config.aggressiveCompressRatio - 0.10));
const em = emergencyCompress(messages, emTargetTokens, tokenArray, aggRemainingTokens);
report.emergencyDeleted = em.deletedCount;
aggRemainingTokens = em.remainingTokens;
compactState.deletedOffloadIds.push(...em.deletedIds);
}
// Step 7: Inject active MMD (after all compression, so position is correct)
const mmdResult = await injectActiveMmd(messages, state, storage, basePath);
report.mmdInjected += mmdResult.injectedCount;
// Step 8: Write compact-state.json (independent file, no lock needed)
report.compactedCount = messages.length;
compactState.lastCompactedAt = new Date().toISOString();
await writeCompactState(storage, basePath, compactState);
// Compute remaining tokens (tracked from aggressive/emergency, no full re-scan)
const remainingTokens = level === "fastpath" || level === "mild"
? totalTokens // no deletion happened
: aggRemainingTokens; // tracked through aggressive → emergency chain
const remainingRatio = contextWindow > 0 ? (remainingTokens / contextWindow).toFixed(2) : "N/A";
// Opik trace: compaction decision
traceServerCompaction({
sessionId,
level,
ratio,
contextWindow,
totalTokensBefore: originalTotalTokens,
totalTokensAfter: remainingTokens,
originalMsgCount: originalCount,
compactedMsgCount: messages.length,
report: report as unknown as Record<string, unknown>,
messages: messages as unknown[],
durationMs: Date.now() - compactionStartMs,
logger: deps.logger,
});
// Step 8: Return
sendJson(res, 200, successEnvelope({ messages, report }, requestId));
deps.logger.info(
`[offload-server] compaction done: ${originalCount}${messages.length} msgs, level=${level}, ` +
`tokens=${originalTotalTokens}${remainingTokens} (${remainingRatio}), ` +
`fp=${fp.replacedCount}r/${fp.deletedCount}d, mild=${report.mildReplacements}, ` +
`agg=${report.aggressiveDeleted}, em=${report.emergencyDeleted}, mmd=${report.mmdInjected}`,
);
}
// ─── Token Array Builder ─────────────────────────────────────────────────────
/** Calibration threshold: if heuristic estimate drifts >15% from API totalTokens, apply linear scaling. */
const CALIBRATION_THRESHOLD = 0.15;
/** Clamp calibration factor to prevent extreme scaling from noisy estimates. */
const CALIBRATION_FACTOR_MIN = 0.5;
const CALIBRATION_FACTOR_MAX = 3.0;
/**
* Build a pre-computed token array for all messages, with optional linear calibration.
* - If messageTokens[i] is available, use it directly (precise value).
* - Otherwise, use estimateMessageTokens (CJK-aware heuristic).
* - If totalTokens is provided and drift > 15%, apply a calibration factor to estimated items.
*/
export function buildTokenArray(
messages: Message[],
totalTokens: number,
messageTokens?: number[],
): number[] {
const raw = messages.map((msg, i) =>
messageTokens && i < messageTokens.length
? messageTokens[i]
: estimateMessageTokens(msg),
);
const rawTotal = raw.reduce((s, v) => s + v, 0);
if (rawTotal <= 0 || totalTokens <= 0) return raw;
const drift = Math.abs(totalTokens - rawTotal) / totalTokens;
if (drift <= CALIBRATION_THRESHOLD) return raw;
// Linear calibration: only scale estimated items, keep precise items unchanged
const factor = Math.max(CALIBRATION_FACTOR_MIN, Math.min(CALIBRATION_FACTOR_MAX, totalTokens / rawTotal));
return raw.map((v, i) =>
messageTokens && i < messageTokens.length
? v
: Math.max(1, Math.round(v * factor)),
);
}
// ─── State Helpers ───────────────────────────────────────────────────────────
async function readOffloadState(storage: StorageAdapter, basePath: string): Promise<OffloadState> {
const raw = await storage.readFile(`${basePath}/state.json`);
if (!raw) return defaultOffloadState();
try {
return { ...defaultOffloadState(), ...JSON.parse(raw) };
} catch {
return defaultOffloadState();
}
}
async function readCompactState(storage: StorageAdapter, basePath: string): Promise<CompactState> {
const raw = await storage.readFile(`${basePath}/compact-state.json`);
if (!raw) return defaultCompactState();
try {
return { ...defaultCompactState(), ...JSON.parse(raw) };
} catch {
return defaultCompactState();
}
}
async function writeCompactState(storage: StorageAdapter, basePath: string, state: CompactState): Promise<void> {
await storage.writeFile(`${basePath}/compact-state.json`, JSON.stringify(state));
}
File diff suppressed because it is too large Load Diff
+189
View File
@@ -0,0 +1,189 @@
/**
* L3 Fast-path re-apply confirmed/deleted offload state to messages.
* Runs on every compaction call as the first step.
*/
import type { OffloadEntry, CompactState } from "../types.js";
import type { Message } from "./helpers.js";
import {
extractToolResultId,
isToolResultMessage,
isOnlyToolUseAssistant,
isAssistantWithToolUse,
extractAllToolUseIds,
replaceWithSummary,
buildOffloadMap,
} from "./helpers.js";
export interface FastPathResult {
replacedCount: number;
deletedCount: number;
}
/**
* Apply fast-path: replace confirmed tool results with summary,
* delete messages whose tool_call_id is in deletedOffloadIds,
* strip orphaned toolCall blocks from mixed assistant messages.
* Mutates messages in place.
*/
export function applyFastPath(
messages: Message[],
entries: OffloadEntry[],
compactState: CompactState,
): FastPathResult {
const offloadMap = buildOffloadMap(entries);
const confirmedSet = new Set(compactState.confirmedOffloadIds);
const deletedSet = new Set(compactState.deletedOffloadIds);
if (confirmedSet.size === 0 && deletedSet.size === 0) {
return { replacedCount: 0, deletedCount: 0 };
}
const indicesToDelete: number[] = [];
let replacedCount = 0;
for (let i = 0; i < messages.length; i++) {
const msg = messages[i];
const tid = extractToolResultId(msg);
// 1. Deleted tool_result — handle Anthropic multi-block user messages carefully
if (tid && deletedSet.has(tid)) {
// Anthropic format: user message may have multiple tool_result blocks
if (msg.role === "user" && Array.isArray(msg.content)) {
const allToolResultIds = msg.content
.filter((b: any) => b?.type === "tool_result" && b.tool_use_id)
.map((b: any) => b.tool_use_id as string);
if (allToolResultIds.length > 1) {
// Multi-block: only remove the blocks that are in deletedSet, keep the rest
const allDeleted = allToolResultIds.every((id: string) => deletedSet.has(id));
if (allDeleted) {
indicesToDelete.push(i);
} else {
// Partial: strip only deleted tool_result blocks
for (let j = msg.content.length - 1; j >= 0; j--) {
const block = msg.content[j];
if (block?.type === "tool_result" && block.tool_use_id && deletedSet.has(block.tool_use_id)) {
msg.content.splice(j, 1);
}
}
}
continue;
}
}
// Single tool_result or non-Anthropic format: delete entire message
indicesToDelete.push(i);
continue;
}
// 2. Confirmed tool_result → replace with summary
if (tid && confirmedSet.has(tid) && isToolResultMessage(msg) && !msg._offloaded) {
const entry = offloadMap.get(tid);
if (entry) {
replaceWithSummary(msg, entry);
replacedCount++;
}
}
// 3. Pure toolCall assistant — all IDs deleted → mark for deletion
if (!tid && isOnlyToolUseAssistant(msg)) {
const tuIds = extractAllToolUseIds(msg);
if (tuIds.length > 0 && tuIds.every((id) => deletedSet.has(id))) {
indicesToDelete.push(i);
}
}
// 4. Mixed assistant (text + toolCall) — strip deleted toolCall blocks
// to prevent orphaned tool_use without matching tool_result (provider 400)
if (!tid && isAssistantWithToolUse(msg) && !isOnlyToolUseAssistant(msg)) {
const content = msg.type === "message" ? msg.message?.content : msg.content;
if (Array.isArray(content)) {
for (let j = content.length - 1; j >= 0; j--) {
const block = content[j];
if ((block?.type === "tool_use" || block?.type === "toolCall") && block.id) {
if (deletedSet.has(block.id)) {
content.splice(j, 1);
}
}
}
}
}
}
// Delete in reverse order to preserve indices
const uniqueIndices = [...new Set(indicesToDelete)].sort((a, b) => b - a);
for (const idx of uniqueIndices) {
messages.splice(idx, 1);
}
// Post-pass: ensure no orphaned tool_use or tool_result after deletion
// Collect all remaining tool_use IDs and tool_result IDs
const remainingToolUseIds = new Set<string>();
const remainingToolResultIds = new Set<string>();
for (const msg of messages) {
for (const id of extractAllToolUseIds(msg)) {
remainingToolUseIds.add(id);
}
if (msg.role === "user" && Array.isArray(msg.content)) {
for (const block of msg.content) {
if (block?.type === "tool_result" && block.tool_use_id) {
remainingToolResultIds.add(block.tool_use_id);
}
}
}
const singleTid = extractToolResultId(msg);
if (singleTid) remainingToolResultIds.add(singleTid);
}
// Remove orphaned tool_use blocks from assistant messages (no matching tool_result)
for (const msg of messages) {
if (!isAssistantWithToolUse(msg)) continue;
const content = msg.type === "message" ? msg.message?.content : msg.content;
if (!Array.isArray(content)) continue;
for (let j = content.length - 1; j >= 0; j--) {
const block = content[j];
if ((block?.type === "tool_use" || block?.type === "toolCall") && block.id) {
if (!remainingToolResultIds.has(block.id)) {
content.splice(j, 1);
}
}
}
}
// Remove orphaned tool_result messages/blocks (no matching tool_use)
for (let i = messages.length - 1; i >= 0; i--) {
const msg = messages[i];
if (msg.role === "user" && Array.isArray(msg.content)) {
for (let j = msg.content.length - 1; j >= 0; j--) {
const block = msg.content[j];
if (block?.type === "tool_result" && block.tool_use_id && !remainingToolUseIds.has(block.tool_use_id)) {
msg.content.splice(j, 1);
}
}
// If user message is now empty, remove it
if (msg.content.length === 0) {
messages.splice(i, 1);
uniqueIndices.push(i); // count it
}
} else {
const singleTid = extractToolResultId(msg);
if (singleTid && !remainingToolUseIds.has(singleTid) && isToolResultMessage(msg)) {
messages.splice(i, 1);
uniqueIndices.push(i);
}
}
}
// Remove empty assistant messages (all tool_use blocks were stripped)
for (let i = messages.length - 1; i >= 0; i--) {
const msg = messages[i];
const role = msg.role ?? msg.message?.role ?? msg.type;
if (role !== "assistant") continue;
const content = msg.type === "message" ? msg.message?.content : msg.content;
if (Array.isArray(content) && content.length === 0) {
messages.splice(i, 1);
uniqueIndices.push(i);
}
}
return { replacedCount, deletedCount: uniqueIndices.length };
}
+297
View File
@@ -0,0 +1,297 @@
/**
* L3 Compaction Message helper utilities.
* Handles multiple message formats (OpenAI, Anthropic, OpenClaw).
*
*
* OpenClaw @mariozechner/pi-ai
*
*
* OpenClaw 使 AgentMessage
*
* 1. UserMessage
* {
* role: "user",
* content: string | ContentBlock[], // string 或 [{type:"text",text:"..."}] 或含图片块
* timestamp: number,
* }
*
* 2. AssistantMessage
* {
* role: "assistant",
* content: [{ type: "text", text: "..." }],
* model: "gpt-5.2",
* stopReason: "stop",
* timestamp: number,
* api: "messages" | "chat" | "responses",
* provider: "anthropic" | "openai" | "google",
* usage: { input, output, totalTokens, ... },
* }
*
* 3. AssistantMessage tool_use / toolCall
* {
* role: "assistant",
* content: [
* { type: "text", text: "I'll read the file..." }, // 可选文本块
* { type: "toolCall", id: "call_abc123", name: "read_file", arguments: { path: "..." } },
* { type: "toolCall", id: "call_def456", name: "exec", arguments: { cmd: "..." } },
* ],
* stopReason: "toolUse",
* ...
* }
* : Anthropic 使 { type: "tool_use", id, name, input }
* OpenClaw { type: "toolCall", id, name, arguments }
* ****
* "tool_use" "toolCall"
*
* 4. ToolResultMessage
* {
* role: "toolResult",
* toolCallId: "call_abc123", // 对应 AssistantMessage 中 toolCall 的 id
* toolName: "read_file",
* content: [{ type: "text", text: "文件内容..." }],
* isError: false,
* timestamp: number,
* details?: any, // 可选的详细信息(不发给 LLM)
* }
*
* 5.
* - AssistantMessage toolCall ToolResultMessage
* - toolCallId
* - tool_result assistant toolCall provider 400
* - assistant toolCall
*
* 6. LLM Provider
* OpenAI: toolResult { role: "tool", tool_call_id: "...", content: "..." }
* Anthropic: toolResult { role: "user", content: [{ type: "tool_result", tool_use_id: "...", content: "..." }] }
* : 本模块处理的是 **** provider
*
*
* 7. 使
* - _offloaded: boolean summary tool_result
* - _mmdContextMessage: string MMD "active" | "history"
* - _mmdInjection: boolean MMD
* - _mmdVersion: string MMD
*
*/
import type { OffloadEntry } from "../types.js";
// ─── Message type aliases ────────────────────────────────────────────────────
// eslint-disable-next-line @typescript-eslint/no-explicit-any
export type Message = Record<string, any>;
// ─── Tool Result ID Extraction ───────────────────────────────────────────────
/**
* Extract the linked tool_use_id from a tool-result message (supports multiple formats).
*/
export function extractToolResultId(msg: Message): string | null {
// OpenAI tool result: { role: "tool", tool_call_id: "..." }
if (msg.tool_call_id) return msg.tool_call_id;
// Anthropic tool_result: { type: "tool_result", tool_use_id: "..." }
if (msg.tool_use_id) return msg.tool_use_id;
// OpenClaw wrapped: { type: "message", message: { id: "...", role: "toolResult" } }
if (msg.type === "message" && msg.message?.id) return msg.message.id;
// Anthropic content block tool_result: { role: "user", content: [{type:"tool_result", tool_use_id}] }
if (msg.role === "user" && Array.isArray(msg.content)) {
for (const block of msg.content) {
if (block?.type === "tool_result" && block.tool_use_id) return block.tool_use_id;
}
}
return null;
}
// ─── Message Type Checks ─────────────────────────────────────────────────────
/**
* Check if a message is a tool result (contains tool output).
*/
export function isToolResultMessage(msg: Message): boolean {
if (msg.role === "tool") return true;
if (msg.type === "tool_result") return true;
const innerRole = msg.message?.role ?? msg.type;
if (innerRole === "toolResult" || innerRole === "tool_result") return true;
// Anthropic user message with tool_result content blocks
if (msg.role === "user" && Array.isArray(msg.content)) {
return msg.content.some((b: any) => b?.type === "tool_result");
}
return false;
}
/**
* Check if a message is an assistant message containing only tool_use blocks.
*/
export function isOnlyToolUseAssistant(msg: Message): boolean {
const role = msg.role ?? msg.message?.role;
if (role !== "assistant") return false;
const content = msg.type === "message" ? msg.message?.content : msg.content;
if (!Array.isArray(content)) return false;
if (content.length === 0) return false;
return content.every((block: any) =>
block?.type === "tool_use" || block?.type === "toolCall",
);
}
/**
* Check if a message is an assistant message that contains tool_use (may also have text).
* Supports both Anthropic format (content blocks) and OpenAI format (tool_calls field).
*/
export function isAssistantWithToolUse(msg: Message): boolean {
const role = msg.role ?? msg.message?.role;
if (role !== "assistant") return false;
// OpenAI format: tool_calls field
if (msg.tool_calls && Array.isArray(msg.tool_calls) && msg.tool_calls.length > 0) return true;
// Anthropic format: content blocks
const content = msg.type === "message" ? msg.message?.content : msg.content;
if (!Array.isArray(content)) return false;
return content.some((block: any) =>
block?.type === "tool_use" || block?.type === "toolCall",
);
}
/**
* Extract all tool_use IDs from an assistant message.
* Supports both Anthropic format (content blocks) and OpenAI format (tool_calls field).
*/
export function extractAllToolUseIds(msg: Message): string[] {
const ids: string[] = [];
// OpenAI format: tool_calls field
if (msg.tool_calls && Array.isArray(msg.tool_calls)) {
for (const tc of msg.tool_calls) {
if (tc.id) ids.push(tc.id);
}
}
// Anthropic format: content blocks
const content = msg.type === "message" ? msg.message?.content : msg.content;
if (Array.isArray(content)) {
for (const block of content) {
if ((block?.type === "tool_use" || block?.type === "toolCall") && block.id) {
ids.push(block.id);
}
}
}
return ids;
}
// ─── Replace With Summary ────────────────────────────────────────────────────
/**
* Replace a tool result message's content with the offload summary.
*/
export function replaceWithSummary(msg: Message, entry: OffloadEntry): void {
const parts = [
`[Offloaded Tool Result | node: ${entry.node_id ?? "N/A"}]`,
`Summary: ${entry.summary}`,
];
if (entry.result_ref) {
parts.push(`原始工具结果已存档,如需查看完整内容请调用 tdai_read_cos(path="${entry.result_ref}")`);
}
const summaryContent = parts.join("\n");
if (msg.type === "message" && msg.message) {
if (Array.isArray(msg.message.content)) {
msg.message.content = [{ type: "text", text: summaryContent }];
} else {
msg.message.content = summaryContent;
}
} else if (msg.role === "user" && Array.isArray(msg.content)) {
// Anthropic tool_result in user content blocks
for (let i = 0; i < msg.content.length; i++) {
if (msg.content[i]?.type === "tool_result") {
msg.content[i].content = summaryContent;
break;
}
}
} else {
if (Array.isArray(msg.content)) {
msg.content = [{ type: "text", text: summaryContent }];
} else {
msg.content = summaryContent;
}
}
msg._offloaded = true;
}
// ─── Offload Map ─────────────────────────────────────────────────────────────
/**
* Build a lookup map from tool_call_id OffloadEntry.
*/
export function buildOffloadMap(entries: OffloadEntry[]): Map<string, OffloadEntry> {
const map = new Map<string, OffloadEntry>();
for (const entry of entries) {
if (entry.tool_call_id) {
map.set(entry.tool_call_id, entry);
}
}
return map;
}
// ─── Token Estimation ────────────────────────────────────────────────────────
/**
* Count CJK characters (CJK Unified Ideographs + Extension A + Compatibility).
*/
function countCjkChars(text: string): number {
let n = 0;
for (const ch of text) {
const c = ch.codePointAt(0)!;
if (
(c >= 0x4e00 && c <= 0x9fff) ||
(c >= 0x3400 && c <= 0x4dbf) ||
(c >= 0xf900 && c <= 0xfaff)
) {
n++;
}
}
return n;
}
/**
* Estimate tokens for a text string using CJK-aware heuristic.
* CJK characters 1 token / 1.7 chars, non-CJK 1 token / 4 chars.
* Aligned with plugin-side estimateL3MixedTokensHeuristic.
*/
function estimateTextTokens(text: string): number {
const cjk = countCjkChars(text);
const rest = Math.max(0, text.length - cjk);
return Math.max(1, Math.ceil(cjk / 1.7 + rest / 4));
}
/**
* Estimate tokens for a message using CJK-aware heuristic (/1.7 + /4).
*/
export function estimateMessageTokens(msg: Message): number {
const content = msg.content ?? msg.message?.content ?? "";
let text: string;
if (typeof content === "string") {
text = content;
} else if (Array.isArray(content)) {
text = content.map((b: any) => (typeof b === "string" ? b : b?.text ?? JSON.stringify(b) ?? "")).join("");
} else {
text = JSON.stringify(content);
}
// Include tool_calls arguments (OpenAI format)
const toolCalls = (msg as any).tool_calls;
if (toolCalls && Array.isArray(toolCalls)) {
for (const tc of toolCalls) {
if (tc.function?.name) text += tc.function.name;
if (tc.function?.arguments) text += tc.function.arguments;
}
}
return estimateTextTokens(text);
}
// ─── MMD Message Markers ─────────────────────────────────────────────────────
export const MMD_CONTEXT_MARKER = "_mmdContextMessage";
export const MMD_INJECTION_MARKER = "_mmdInjection";
/**
* Check if a message is a MMD-related message (should be preserved during compression).
*/
export function isMmdMessage(msg: Message): boolean {
return !!msg[MMD_CONTEXT_MARKER] || !!msg[MMD_INJECTION_MARKER];
}
+430
View File
@@ -0,0 +1,430 @@
/**
* L3 MMD Injector inject active/history MMD into messages.
* Handles version dedup and insertion point calculation.
*/
import type { StorageAdapter } from "../../core/storage/adapter.js";
import type { OffloadEntry, OffloadState } from "../types.js";
import type { Message } from "./helpers.js";
import { MMD_CONTEXT_MARKER, MMD_INJECTION_MARKER, buildOffloadMap } from "./helpers.js";
// ─── Active MMD Injection ────────────────────────────────────────────────────
export interface MmdInjectionResult {
injectedCount: number;
mmdTokensEstimate: number;
}
/**
* Inject the active MMD into messages. Performs version dedup.
*/
export async function injectActiveMmd(
messages: Message[],
state: OffloadState,
storage: StorageAdapter,
basePath: string,
): Promise<MmdInjectionResult> {
// Resolve the MMD file to inject: use activeMmdFile, or fallback to the latest .mmd
let mmdFile = state.activeMmdFile;
if (!mmdFile) {
// Fallback: find the most recent .mmd file (sorted by prefix number, highest = latest)
const allMmds = await storage.readdirNames(`${basePath}/mmds/`, ".mmd");
if (allMmds.length > 0) {
// Sort descending by prefix number (e.g., "006-current-task.mmd" → 6)
allMmds.sort((a, b) => {
const numA = parseInt(a.split("-")[0], 10) || 0;
const numB = parseInt(b.split("-")[0], 10) || 0;
return numB - numA;
});
// Pick the first non-empty one
for (const candidate of allMmds) {
const content = await storage.readFile(`${basePath}/mmds/${candidate}`);
if (content && content.trim().length > 0) {
mmdFile = candidate;
break;
}
}
}
}
if (!mmdFile) {
removeMmdMessages(messages, "active");
return { injectedCount: 0, mmdTokensEstimate: 0 };
}
const mmdContent = await storage.readFile(`${basePath}/mmds/${mmdFile}`);
if (!mmdContent) {
removeMmdMessages(messages, "active");
return { injectedCount: 0, mmdTokensEstimate: 0 };
}
const newVersion = hashContent(mmdContent);
// Version dedup: check existing MMD message version and position
const existingIdx = messages.findIndex((m) => m[MMD_CONTEXT_MARKER] === "active");
if (existingIdx >= 0 && messages[existingIdx]._mmdVersion === newVersion) {
// Content unchanged — but still need to ensure correct position
// (aggressive compression may have shifted it)
const correctIdx = findActiveMmdInsertionPoint(
messages.filter((m) => m[MMD_CONTEXT_MARKER] !== "active"),
);
// Check if position is already correct (accounting for removal offset)
const effectiveCorrectIdx = correctIdx <= existingIdx ? correctIdx : correctIdx;
if (existingIdx === effectiveCorrectIdx) {
return { injectedCount: 0, mmdTokensEstimate: 0 };
}
// Reposition: remove and re-insert at correct point
const [mmdMsg] = messages.splice(existingIdx, 1);
const newInsertIdx = findActiveMmdInsertionPoint(messages);
messages.splice(newInsertIdx, 0, mmdMsg);
return { injectedCount: 0, mmdTokensEstimate: 0 };
}
// Remove old active MMD message
removeMmdMessages(messages, "active");
// Build MMD text
const mmdText = buildActiveMmdText(mmdFile, mmdContent);
const mmdMsg: Message = {
role: "user",
content: mmdText,
[MMD_CONTEXT_MARKER]: "active",
_mmdVersion: newVersion,
_mmdFilename: mmdFile,
};
// Insert at calculated point
const insertIdx = findActiveMmdInsertionPoint(messages);
messages.splice(insertIdx, 0, mmdMsg);
return { injectedCount: 1, mmdTokensEstimate: Math.ceil(mmdContent.length / 4) };
}
// ─── History MMD Injection ───────────────────────────────────────────────────
export interface HistoryMmdResult {
injectedCount: number;
mmdFiles: string[];
totalTokensEstimate: number;
}
/**
* Inject history MMD files for entries that were deleted during aggressive compression.
*/
export async function injectHistoryMmds(
messages: Message[],
deletedIds: string[],
entries: OffloadEntry[],
state: OffloadState,
storage: StorageAdapter,
basePath: string,
tokenBudget: number,
): Promise<HistoryMmdResult> {
if (deletedIds.length === 0) {
return { injectedCount: 0, mmdFiles: [], totalTokensEstimate: 0 };
}
// 1. Find MMD prefixes from deleted entries' node_ids
const offloadMap = buildOffloadMap(entries);
const mmdPrefixes = new Set<string>();
for (const id of deletedIds) {
const entry = offloadMap.get(id);
if (entry?.node_id) {
const prefix = entry.node_id.split("-")[0];
if (prefix) mmdPrefixes.add(prefix);
}
}
if (mmdPrefixes.size === 0) {
return { injectedCount: 0, mmdFiles: [], totalTokensEstimate: 0 };
}
// 2. Find matching history MMD files (exclude active)
const allMmds = await storage.readdirNames(`${basePath}/mmds/`, ".mmd");
const candidates = allMmds.filter((f) => {
const prefix = f.split("-")[0];
return mmdPrefixes.has(prefix) && f !== state.activeMmdFile;
});
if (candidates.length === 0) {
return { injectedCount: 0, mmdFiles: [], totalTokensEstimate: 0 };
}
// 3. Read and inject (respecting token budget), most recent first
candidates.reverse();
const injected: Message[] = [];
const mmdFiles: string[] = [];
let usedTokens = 0;
for (const filename of candidates) {
const content = await storage.readFile(`${basePath}/mmds/${filename}`);
if (!content) continue;
const text = buildHistoryMmdText(filename, content);
const tokens = Math.ceil(text.length / 4);
if (usedTokens + tokens > tokenBudget) continue;
injected.push({
role: "user",
content: text,
[MMD_INJECTION_MARKER]: true,
_mmdFilename: filename,
});
mmdFiles.push(filename);
usedTokens += tokens;
}
if (injected.length === 0) {
return { injectedCount: 0, mmdFiles: [], totalTokensEstimate: 0 };
}
// 4. Remove old history MMD injections and insert new ones
removeExistingMmdInjections(messages);
const insertIdx = findHistoryMmdInsertionPoint(messages);
// Reverse back to chronological order (oldest first)
injected.reverse();
mmdFiles.reverse();
messages.splice(insertIdx, 0, ...injected);
return { injectedCount: injected.length, mmdFiles, totalTokensEstimate: usedTokens };
}
// ─── Insertion Point Calculation ─────────────────────────────────────────────
/**
* Find insertion point for active MMD.
* Strategy: insert after the latest user message in the second half.
* Guard: don't split tool_call / tool_result pairs.
*/
export function findActiveMmdInsertionPoint(messages: Message[]): number {
if (messages.length <= 2) {
const idx = Math.min(1, messages.length);
return adjustForToolCallPair(messages, idx);
}
let latestUserIdx = -1;
for (let i = messages.length - 1; i >= 0; i--) {
const role = messages[i].role ?? messages[i].message?.role ?? messages[i].type;
if (role === "user" && !messages[i][MMD_CONTEXT_MARKER] && !messages[i][MMD_INJECTION_MARKER]) {
latestUserIdx = i;
break;
}
}
let insertIdx: number;
if (latestUserIdx >= 0) {
// Insert relative to latest user message:
// - If it's the last message → insert before it (user prompt stays last)
// - Otherwise → insert after it (between user and tool loop, matching plugin behavior)
if (latestUserIdx === messages.length - 1) {
insertIdx = latestUserIdx;
} else {
insertIdx = latestUserIdx + 1;
}
} else {
// No user message found — fallback: before the trailing tool loop
let loopStart = messages.length;
for (let i = messages.length - 1; i >= 0; i--) {
const role = messages[i].role ?? messages[i].message?.role ?? messages[i].type;
if (messages[i][MMD_CONTEXT_MARKER] || messages[i][MMD_INJECTION_MARKER]) continue;
if (role === "toolResult" || role === "tool" || role === "assistant") {
loopStart = i;
} else {
break;
}
}
const maxDistFromTail = 30;
const minInsertIdx = Math.max(1, messages.length - maxDistFromTail);
insertIdx = Math.max(loopStart, minInsertIdx);
}
// Guard: don't insert between assistant(tool_use) and its tool_result
insertIdx = adjustForToolCallPair(messages, insertIdx);
// Hard guard: never insert before system message
if (insertIdx === 0 && messages.length > 0) {
const firstRole = messages[0].role ?? messages[0].message?.role ?? messages[0].type;
if (firstRole === "system") {
insertIdx = 1;
}
}
return insertIdx;
}
/**
* Find insertion point for history MMD (before active MMD).
*/
export function findHistoryMmdInsertionPoint(messages: Message[]): number {
for (let i = 0; i < messages.length; i++) {
if (messages[i][MMD_CONTEXT_MARKER] === "active") return i;
}
return findActiveMmdInsertionPoint(messages);
}
// ─── MMD Message Management ─────────────────────────────────────────────────
/**
* Remove MMD messages by type ("active", "history", or all).
*/
export function removeMmdMessages(messages: Message[], type?: "active" | "history"): number {
let removed = 0;
for (let i = messages.length - 1; i >= 0; i--) {
if (type === "active" && messages[i][MMD_CONTEXT_MARKER] === "active") {
messages.splice(i, 1);
removed++;
} else if (type === "history" && messages[i][MMD_INJECTION_MARKER]) {
messages.splice(i, 1);
removed++;
} else if (!type && (messages[i][MMD_CONTEXT_MARKER] || messages[i][MMD_INJECTION_MARKER])) {
messages.splice(i, 1);
removed++;
}
}
return removed;
}
/**
* Remove existing history MMD injection messages.
*/
export function removeExistingMmdInjections(messages: Message[]): number {
let removed = 0;
for (let i = messages.length - 1; i >= 0; i--) {
if (messages[i][MMD_INJECTION_MARKER]) {
messages.splice(i, 1);
removed++;
}
}
return removed;
}
// ─── Text Builders ───────────────────────────────────────────────────────────
/**
* Build active MMD injection text.
*/
export function buildActiveMmdText(filename: string, mmdContent: string): string {
let taskGoal = "";
const metaMatch = mmdContent.match(/^%%\{\s*(.*?)\s*\}%%/);
if (metaMatch) {
try {
const meta = JSON.parse(`{${metaMatch[1]}}`);
taskGoal = meta.taskGoal || "";
} catch { /* ignore */ }
}
return [
`<current_task_context>`,
`【当前活跃任务的mermaid流程图】这是你最近正在执行的任务的阶段性记录。`,
taskGoal ? `**任务目标:** ${taskGoal}` : "",
`**任务文件:** ${filename}`,
"```mermaid",
mmdContent,
"```",
`标记为 "doing" 的节点是近期焦点,"done" 的已完成。请参考此保持方向感,避免重复已完成的工作。`,
`</current_task_context>`,
]
.filter((line) => line !== "")
.join("\n");
}
/**
* Build history MMD injection text.
*/
export function buildHistoryMmdText(filename: string, mmdContent: string): string {
let taskGoal = "";
const metaMatch = mmdContent.match(/^%%\{\s*(.*?)\s*\}%%/);
if (metaMatch) {
try {
const meta = JSON.parse(`{${metaMatch[1]}}`);
taskGoal = meta.taskGoal || "";
} catch { /* ignore */ }
}
return [
`<history_task_context>`,
`【历史任务记录】以下是此前完成的任务的概要。`,
taskGoal ? `**任务目标:** ${taskGoal}` : "",
`**任务文件:** ${filename}`,
"```mermaid",
mmdContent,
"```",
`</history_task_context>`,
]
.filter((line) => line !== "")
.join("\n");
}
// ─── Internal Helpers ────────────────────────────────────────────────────────
function hasToolResultContent(msg: Message): boolean {
const content = msg?.content;
if (!Array.isArray(content)) return false;
return content.some(
(block: any) =>
typeof block === "object" &&
block !== null &&
(block.type === "tool_result" || block.type === "toolResult"),
);
}
function hasToolUseContent(msg: Message): boolean {
const content = msg?.content;
if (!Array.isArray(content)) return false;
return content.some(
(block: any) =>
typeof block === "object" &&
block !== null &&
(block.type === "tool_use" || block.type === "toolUse"),
);
}
function adjustForToolCallPair(messages: Message[], insertIdx: number): number {
if (insertIdx <= 0 || insertIdx >= messages.length) return insertIdx;
const msgAtIdx = messages[insertIdx];
const role = msgAtIdx?.role ?? msgAtIdx?.message?.role ?? msgAtIdx?.type;
// Check both role-level (OpenAI: role="tool") and content-level (Anthropic: content[].type="tool_result")
const isToolResult =
role === "tool" ||
role === "toolResult" ||
role === "tool_result" ||
(role === "user" && hasToolResultContent(msgAtIdx));
if (isToolResult) {
// Walk back to before the assistant tool_use
let i = insertIdx - 1;
while (i >= 0) {
const r = messages[i].role ?? messages[i].message?.role ?? messages[i].type;
if (r === "assistant" && hasToolUseContent(messages[i])) {
return i;
}
if (r === "assistant") {
return i;
}
const prevIsToolResult =
r === "tool" ||
r === "toolResult" ||
r === "tool_result" ||
(r === "user" && hasToolResultContent(messages[i]));
if (!prevIsToolResult) break;
i--;
}
}
// Also check: don't insert right after an assistant with tool_use (before its tool_result)
if (insertIdx > 0) {
const prevMsg = messages[insertIdx - 1];
const prevRole = prevMsg?.role ?? prevMsg?.message?.role ?? prevMsg?.type;
if (prevRole === "assistant" && hasToolUseContent(prevMsg)) {
return insertIdx - 1;
}
}
return insertIdx;
}
function hashContent(s: string): string {
let hash = 0;
for (let i = 0; i < s.length; i++) {
hash = ((hash << 5) - hash + s.charCodeAt(i)) | 0;
}
return Math.abs(hash).toString(36).padStart(6, "0").slice(0, 6);
}
+262
View File
@@ -0,0 +1,262 @@
/**
* Offload Ingest Handler fast path: COS write + task enqueue.
* Branch A: toolPairs only write pending.jsonl, enqueue async L1
* Branch B: recentMessages only (no toolPairs) L1.5 path
* Branch C: toolPairs + recentMessages write pending + cache recentMessages, trigger L1 (skip L1.5)
*/
import type http from "node:http";
import type { StorageAdapter } from "../core/storage/adapter.js";
import type { IStateBackend, TaskPayload } from "../core/state/types.js";
import type { OffloadExecutorConfig, OffloadState } from "./types.js";
import { defaultOffloadState } from "./types.js";
import { IngestRequestSchema } from "./schemas.js";
import { serializeJsonl } from "./parsers/json-utils.js";
import { buildOffloadBasePath } from "./session-utils.js";
// ─── Per-session in-process mutex for COS append serialization ───
// Layer 1 (local): ensures at most one inflight appendFile per session within
// the same Node.js process (no I/O overhead, zero-latency queue).
// Layer 2 (distributed): stateBackend.acquireLock across multiple offload server
// instances prevents concurrent AppendObject calls to the same COS key.
const pendingMutexes = new Map<string, Promise<void>>();
async function withSessionMutex<T>(sessionKey: string, fn: () => Promise<T>): Promise<T> {
const prev = pendingMutexes.get(sessionKey) ?? Promise.resolve();
let resolve!: () => void;
const next = new Promise<void>((r) => { resolve = r; });
pendingMutexes.set(sessionKey, next);
try {
await prev;
return await fn();
} finally {
resolve();
if (pendingMutexes.get(sessionKey) === next) {
pendingMutexes.delete(sessionKey);
}
}
}
/** Max attempts to acquire the distributed lock. */
const APPEND_LOCK_MAX_ATTEMPTS = 10;
/** TTL for the distributed append lock (ms). Short-lived since append is fast. */
const APPEND_LOCK_TTL_MS = 5000;
/** Delay between lock acquisition retries (ms). Uses exponential backoff: base * 2^attempt. */
const APPEND_LOCK_RETRY_BASE_MS = 50;
export interface IngestDeps {
storage: StorageAdapter;
stateBackend?: IStateBackend;
config: OffloadExecutorConfig;
logger: { info: (...args: unknown[]) => void; warn: (...args: unknown[]) => void };
}
export async function handleIngest(
req: http.IncomingMessage,
res: http.ServerResponse,
auth: { serviceId: string },
deps: IngestDeps,
requestId: string,
parseJsonBody: <T>(req: http.IncomingMessage) => Promise<T>,
sendJson: (res: http.ServerResponse, status: number, body: unknown) => void,
successEnvelope: <T>(data: T, requestId: string) => unknown,
errorEnvelope: (code: number, message: string, requestId: string) => unknown,
): Promise<void> {
const body = await parseJsonBody(req);
const parsed = IngestRequestSchema.safeParse(body);
if (!parsed.success) {
sendJson(res, 400, errorEnvelope(400, parsed.error.message, requestId));
return;
}
const { session_id: sessionId, tool_pairs: toolPairs, prompt, recent_messages: recentMessages } = parsed.data;
const { storage, stateBackend, config } = deps;
const basePath = buildOffloadBasePath(sessionId);
// Build context string from structured prompt + recentMessages (for L1/L1.5)
let contextText: string | undefined;
if (prompt || (recentMessages && recentMessages.length > 0)) {
const parts: string[] = [];
if (recentMessages && recentMessages.length > 0) {
parts.push("历史消息,可作为参考:");
for (const m of recentMessages) {
parts.push(`[${m.role === "user" ? "User" : "Assistant"}]: ${m.content}`);
}
}
if (prompt) {
parts.push(`\n最新user message\n[User]: ${prompt}`);
}
contextText = parts.join("\n");
}
// ─── Session skip filter (shared by L1 and L1.5) ───
const INTERNAL_SESSION_RE = /memory-.*-session-\d+/;
const shouldSkipSession = INTERNAL_SESSION_RE.test(sessionId) || sessionId.includes("subagent");
// ─── Branch L1: toolPairs non-empty → write pending + trigger L1 ───
if (toolPairs.length > 0) {
if (shouldSkipSession) {
deps.logger.info(`[offload-server] ingest: L1 skipped (session=${sessionId})`);
sendJson(res, 200, successEnvelope({}, requestId));
return;
}
// Save context for L1 executor (if available)
if (contextText) {
await storage.writeFile(`${basePath}/recent-context.txt`, contextText);
}
const pendingPath = `${basePath}/pending.jsonl`;
// Map API snake_case → internal camelCase for JSONL storage
const camelPairs = toolPairs.map((tp: Record<string, unknown>) => ({
toolName: tp.tool_name,
toolCallId: tp.tool_call_id,
params: tp.params,
result: tp.result,
error: tp.error,
timestamp: tp.timestamp,
durationMs: tp.duration_ms,
}));
const lines = serializeJsonl(camelPairs);
// Two-layer serialization to prevent COS AppendPositionErr:
// Layer 1 (local mutex): queues concurrent requests within this process.
// Layer 2 (distributed lock via stateBackend): prevents races across server instances.
// If the lock cannot be acquired, return 409 so the client retries (NOT proceed without lock).
const lockAcquired = await withSessionMutex(pendingPath, async () => {
const lockKey = `offload-pending:${auth.serviceId}:${sessionId}`;
const lockOwner = requestId;
let locked = false;
if (stateBackend) {
for (let attempt = 0; attempt < APPEND_LOCK_MAX_ATTEMPTS; attempt++) {
locked = await stateBackend.acquireLock(lockKey, lockOwner, APPEND_LOCK_TTL_MS);
if (locked) break;
// Exponential backoff: 50, 100, 200, 400, 800, 1600... capped at 2000ms
const delay = Math.min(APPEND_LOCK_RETRY_BASE_MS * 2 ** attempt, 2000);
await new Promise((r) => setTimeout(r, delay));
}
if (!locked) {
deps.logger.warn(`[offload-server] ingest: append lock failed after ${APPEND_LOCK_MAX_ATTEMPTS} attempts (session=${sessionId}), returning 409`);
return false;
}
}
try {
await storage.appendFile(pendingPath, lines);
if (stateBackend) {
const raw = await storage.readFile(pendingPath);
const lineCount = raw ? raw.split("\n").filter(Boolean).length : 0;
if (lineCount >= config.forceTriggerThreshold) {
const task: TaskPayload = {
id: `offload-l1-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`,
type: "offload-l1" as TaskPayload["type"],
instanceId: auth.serviceId,
sessionId,
priority: 0,
data: { sessionId, instanceId: auth.serviceId },
createdAt: Date.now(),
};
await stateBackend.enqueueTask(task);
deps.logger.info(`[offload-server] ingest: ${toolPairs.length} pairs, triggered=true (lines=${lineCount})`);
} else {
await stateBackend.setTimerIfEarlier(
auth.serviceId,
`offload-l1:${auth.serviceId}:${sessionId}`,
Date.now() + config.pendingMaxAgeSeconds * 1000,
);
deps.logger.info(`[offload-server] ingest: ${toolPairs.length} pairs, lines=${lineCount}/${config.forceTriggerThreshold}, timer set`);
}
}
return true;
} finally {
if (locked && stateBackend) {
await stateBackend.releaseLock(lockKey, lockOwner);
}
}
});
if (!lockAcquired) {
sendJson(res, 409, errorEnvelope(409, "Concurrent write conflict, please retry", requestId));
return;
}
sendJson(res, 200, successEnvelope({}, requestId));
return;
}
// ─── Branch L1.5: toolPairs empty + prompt → task judgment ───
// Skip L1.5 for inter-session messages, internal sessions, and system prompts
const shouldSkipL15 = shouldSkipSession || !!(
prompt && (
prompt.startsWith("[Inter-session message]") ||
prompt.startsWith("Pre-compaction")
)
);
if (prompt && stateBackend && !shouldSkipL15) {
const lockKey = `offload-state:${auth.serviceId}:${sessionId}`;
const lockOwner = requestId;
let locked = false;
// Use exponential backoff, same policy as L1 append lock.
for (let attempt = 0; attempt < APPEND_LOCK_MAX_ATTEMPTS; attempt++) {
locked = await stateBackend.acquireLock(lockKey, lockOwner, APPEND_LOCK_TTL_MS);
if (locked) break;
const delay = Math.min(APPEND_LOCK_RETRY_BASE_MS * 2 ** attempt, 2000);
await new Promise((r) => setTimeout(r, delay));
}
if (!locked) {
// Do NOT proceed without the lock: a concurrent writer would overwrite state.json
// and silently drop this boundary, causing the L1.5 executor to skip the task
// (findIndex returns -1 → return without error).
deps.logger.warn(`[offload-server] ingest: L1.5 state lock failed after ${APPEND_LOCK_MAX_ATTEMPTS} attempts (session=${sessionId}), returning 409`);
sendJson(res, 409, errorEnvelope(409, "Concurrent write conflict, please retry", requestId));
return;
}
try {
const state = await readState(storage, basePath);
const boundaryTimestamp = new Date().toISOString();
state.boundaries.push({
targetMmd: "_pending",
timestamp: boundaryTimestamp,
});
await writeState(storage, basePath, state);
const task: TaskPayload = {
id: `offload-l15-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`,
type: "offload-l15" as TaskPayload["type"],
instanceId: auth.serviceId,
sessionId,
priority: 0,
data: { sessionId, recentMessages: contextText, boundaryTimestamp, instanceId: auth.serviceId },
createdAt: Date.now(),
};
await stateBackend.enqueueTask(task);
deps.logger.info(`[offload-server] ingest: L1.5 triggered, boundaryTs=${boundaryTimestamp}, prompt=${prompt.slice(0, 200)}`);
} finally {
await stateBackend.releaseLock(lockKey, lockOwner);
}
}
// ─── Fast return ───
sendJson(res, 200, successEnvelope({}, requestId));
}
// ─── State helpers (duplicated intentionally to keep ingest-handler self-contained) ──
async function readState(storage: StorageAdapter, basePath: string): Promise<OffloadState> {
const raw = await storage.readFile(`${basePath}/state.json`);
if (!raw) return defaultOffloadState();
try {
return { ...defaultOffloadState(), ...JSON.parse(raw) };
} catch {
return defaultOffloadState();
}
}
async function writeState(storage: StorageAdapter, basePath: string, state: OffloadState): Promise<void> {
await storage.writeFile(`${basePath}/state.json`, JSON.stringify(state));
}
+90
View File
@@ -0,0 +1,90 @@
/**
* Offload MMD Query Handler returns MMD files.
* Body params: sessionId (required), limit (optional, default=all)
* When limit=1, only returns the active MMD.
*/
import type http from "node:http";
import type { StorageAdapter } from "../core/storage/adapter.js";
import type { OffloadState } from "./types.js";
import { createHash } from "node:crypto";
import { buildOffloadBasePath } from "./session-utils.js";
function hashContent(content: string): string {
return createHash("md5").update(content).digest("hex").slice(0, 12);
}
export async function handleMmdQuery(
_req: http.IncomingMessage,
res: http.ServerResponse,
_auth: { serviceId: string },
storage: StorageAdapter,
requestId: string,
sendJson: (res: http.ServerResponse, status: number, body: unknown) => void,
successEnvelope: <T>(data: T, requestId: string) => unknown,
errorEnvelope: (code: number, message: string, requestId: string) => unknown,
sessionId: string,
limit?: number,
): Promise<void> {
if (!sessionId) {
sendJson(res, 400, errorEnvelope(400, "missing sessionId", requestId));
return;
}
const basePath = buildOffloadBasePath(sessionId);
// Read state
const stateRaw = await storage.readFile(`${basePath}/state.json`);
let state: Partial<OffloadState> = {};
if (stateRaw) {
try {
state = JSON.parse(stateRaw);
} catch {
// ignore
}
}
const mmdsPrefix = `${basePath}/mmds/`;
const mmds: Array<{
filename: string;
content: string;
version: string;
}> = [];
if (limit === 1 && state.activeMmdFile) {
// Fast path: only return the active MMD
const content = await storage.readFile(`${mmdsPrefix}${state.activeMmdFile}`) ?? "";
mmds.push({
filename: state.activeMmdFile,
content,
version: content ? hashContent(content) : "",
});
} else {
// Return all (or up to limit)
const mmdFiles = await storage.readdirNames(mmdsPrefix, ".mmd");
const filesToRead = limit && limit > 0 ? mmdFiles.slice(0, limit) : mmdFiles;
const readResults = await Promise.all(
filesToRead.map(async (filename) => {
const content = await storage.readFile(`${mmdsPrefix}${filename}`) ?? "";
return {
filename,
content,
version: content ? hashContent(content) : "",
};
}),
);
mmds.push(...readResults);
}
sendJson(
res,
200,
successEnvelope(
{
mmds,
currentMmd: state.activeMmdFile ?? null,
},
requestId,
),
);
}
+745
View File
@@ -0,0 +1,745 @@
/**
* Offload Task Executor async L1/L1.5/L2 execution via PipelineWorker.
*/
import type { StorageAdapter } from "../core/storage/adapter.js";
import type { IStateBackend, TaskPayload } from "../core/state/types.js";
import type {
OffloadEntry,
OffloadState,
OffloadExecutorConfig,
L2ParsedResponse,
MmdMeta,
ToolPair,
} from "./types.js";
import { defaultOffloadState, defaultOffloadConfig } from "./types.js";
import { parseJsonl, serializeJsonl } from "./parsers/json-utils.js";
import { parseL1Response } from "./parsers/l1-parser.js";
import { parseL15Response } from "./parsers/l15-parser.js";
import { parseL2Response } from "./parsers/l2-parser.js";
import { L1_SYSTEM_PROMPT, buildL1UserPrompt } from "./prompts/l1-prompt.js";
import { L15_SYSTEM_PROMPT, buildL15UserPrompt } from "./prompts/l15-prompt.js";
import { L2_SYSTEM_PROMPT, buildL2UserPrompt } from "./prompts/l2-prompt.js";
import { handleTaskTransition, extractMmdMeta } from "./task-transition.js";
import { buildOffloadBasePath } from "./session-utils.js";
import { traceServerModelIo, traceServerTaskDecision } from "./opik-tracer.js";
// ─── LLM Client Interface ────────────────────────────────────────────────────
export interface LlmClient {
chat(params: {
model: string;
messages: Array<{ role: "system" | "user"; content: string }>;
temperature: number;
max_tokens: number;
timeoutMs?: number;
}): Promise<string>;
}
// ─── Logger Interface ────────────────────────────────────────────────────────
export interface ExecutorLogger {
info: (...args: unknown[]) => void;
warn: (...args: unknown[]) => void;
error: (...args: unknown[]) => void;
}
// ─── Executor Deps ───────────────────────────────────────────────────────────
export interface OffloadExecutorDeps {
resolveStorage: (instanceId: string) => Promise<StorageAdapter | undefined>;
llmClient: LlmClient;
stateBackend: IStateBackend;
config?: OffloadExecutorConfig;
logger: ExecutorLogger;
}
// ─── Executor Class ──────────────────────────────────────────────────────────
export class OffloadTaskExecutor {
private deps: OffloadExecutorDeps;
private config: OffloadExecutorConfig;
constructor(deps: OffloadExecutorDeps) {
this.deps = deps;
this.config = deps.config ?? defaultOffloadConfig();
}
// ═══════════════════════════════════════════════════════════════════════════
// L1: Summarize pending ToolPairs → OffloadEntry[]
// ═══════════════════════════════════════════════════════════════════════════
async executeOffloadL1(task: TaskPayload, _signal?: AbortSignal): Promise<void> {
const startMs = Date.now();
const sessionId = this.extractSessionId(task);
if (!sessionId) return;
const storage = await this.resolveStorageOrThrow(task.instanceId);
const basePath = buildOffloadBasePath(sessionId);
const pendingPath = `${basePath}/pending.jsonl`;
// 1. Claim pending via rename (atomic: new ingest appends to fresh pending.jsonl)
const processingPath = `${basePath}/pending-processing-${task.id}.jsonl`;
const pendingRaw = await storage.readFile(pendingPath);
if (!pendingRaw || !pendingRaw.trim()) return;
try {
await storage.rename(pendingPath, processingPath);
} catch {
// rename failed (file gone — another L1 claimed it, or doesn't exist)
return;
}
// Re-read from processing file (rename succeeded, this is our exclusive copy)
const claimedRaw = await storage.readFile(processingPath);
if (!claimedRaw || !claimedRaw.trim()) {
await storage.unlink(processingPath);
return;
}
let toolPairs = parseJsonl<Record<string, unknown>>(claimedRaw, (line, err) => {
this.deps.logger.warn(`[offload-server] L1: bad JSONL line in pending: ${line}`, err);
});
if (toolPairs.length === 0) {
await storage.unlink(processingPath);
return;
}
// 2. Dedup: remove toolPairs whose toolCallId already exists in entries.jsonl
const entriesPath = `${basePath}/entries.jsonl`;
const existingEntriesRaw = await storage.readFile(entriesPath);
if (existingEntriesRaw) {
const existingIds = new Set<string>();
for (const e of parseJsonl<OffloadEntry>(existingEntriesRaw)) {
if (e.tool_call_id) existingIds.add(e.tool_call_id);
}
const before = toolPairs.length;
toolPairs = toolPairs.filter((tp) => {
const id = tp.toolCallId as string;
return !id || !existingIds.has(id);
});
if (toolPairs.length < before) {
this.deps.logger.info(`[offload-server] L1: dedup removed ${before - toolPairs.length}/${before} duplicate toolPairs`);
}
if (toolPairs.length === 0) {
await storage.unlink(processingPath);
return;
}
}
// 3. Build prompt & call LLM
const recentContext = await storage.readFile(`${basePath}/recent-context.txt`) ?? "";
const userPrompt = buildL1UserPrompt(recentContext, toolPairs as unknown as ToolPair[]);
let newEntries: OffloadEntry[];
const l1LlmStart = Date.now();
let l1RawResponse: string | undefined;
try {
const response = await this.deps.llmClient.chat({
model: this.config.l1Model,
messages: [
{ role: "system", content: L1_SYSTEM_PROMPT },
{ role: "user", content: userPrompt },
],
temperature: this.config.l1Temperature,
max_tokens: this.config.l1MaxTokens,
timeoutMs: this.config.l1TimeoutMs,
});
l1RawResponse = response;
newEntries = parseL1Response(response);
traceServerModelIo({
sessionId,
stage: "L1",
model: this.config.l1Model,
systemPrompt: L1_SYSTEM_PROMPT,
userPrompt,
responseContent: response,
status: "ok",
durationMs: Date.now() - l1LlmStart,
logger: this.deps.logger,
});
} catch (err) {
traceServerModelIo({
sessionId,
stage: "L1",
model: this.config.l1Model,
systemPrompt: L1_SYSTEM_PROMPT,
userPrompt,
responseContent: l1RawResponse ?? "",
status: "error",
errorMessage: String(err),
durationMs: Date.now() - l1LlmStart,
logger: this.deps.logger,
});
this.deps.logger.error(`[offload-server] L1 LLM failed:`, err);
throw err; // Let Worker retry (processing file remains for re-claim)
}
// 4. Ensure all toolPairs are covered — fallback for missing entries
const parsedIds = new Set(newEntries.map((e) => e.tool_call_id));
// Build toolCallId → original timestamp map for reliable boundary matching
const tpTimestampMap = new Map<string, string>();
for (const tp of toolPairs) {
const id = tp.toolCallId as string;
if (id && tp.timestamp) tpTimestampMap.set(id, tp.timestamp as string);
}
// Overwrite entry timestamps with original tool pair timestamps (don't trust LLM output)
for (const entry of newEntries) {
const origTs = tpTimestampMap.get(entry.tool_call_id);
if (origTs) entry.timestamp = origTs;
}
for (const tp of toolPairs) {
const id = tp.toolCallId as string;
if (id && !parsedIds.has(id)) {
newEntries.push({
tool_call_id: id,
tool_call: (tp.toolName as string) ?? "",
summary: "[L1 parse incomplete]",
timestamp: (tp.timestamp as string) ?? new Date().toISOString(),
score: 2,
node_id: null,
});
}
}
// 5. Write refs: store original tool result content for each pair
for (const tp of toolPairs) {
const id = tp.toolCallId as string;
if (!id) continue;
const result = tp.result ?? tp.error ?? "";
const resultStr = typeof result === "string" ? result : JSON.stringify(result, null, 2);
if (!resultStr || resultStr.length < 20) continue; // skip trivially short results
const toolName = (tp.toolName as string) ?? "unknown";
const timestamp = (tp.timestamp as string) ?? new Date().toISOString();
const header = `# Tool Result: ${toolName}\n\n**tool_call_id:** ${id}\n**Timestamp:** ${timestamp}\n\n---\n\n`;
const refPath = `${basePath}/refs/${id}.md`;
await storage.writeFile(refPath, header + resultStr);
// Set result_ref on the matching entry (full relative path for tdai_read_cos)
const entry = newEntries.find((e) => e.tool_call_id === id);
if (entry) entry.result_ref = `${basePath}/refs/${id}.md`;
}
// 6. Write entries (atomic append) + delete processing file
await storage.appendFile(entriesPath, serializeJsonl(newEntries));
await storage.unlink(processingPath);
// 7. Check L2 trigger: group null entries by boundary targetMmd
const state = await this.readState(storage, basePath);
const allEntriesRaw = await storage.readFile(entriesPath);
if (allEntriesRaw && state.boundaries.length > 0) {
const allEntries = parseJsonl<OffloadEntry>(allEntriesRaw, (line, err) => {
this.deps.logger.warn(`[offload-server] L1: bad JSONL line in entries: ${line}`, err);
});
const nodeMapping = await this.readNodeMapping(storage, basePath);
// Group null entries by resolved targetMmd
const nullByMmd = new Map<string, number>();
for (const e of allEntries) {
if (this.getEffectiveNodeId(e, nodeMapping) !== null || !e.timestamp) continue;
const boundary = this.findBoundaryByTimestamp(state.boundaries, e.timestamp);
if (!boundary) continue; // no boundary → ignore
if (boundary.targetMmd === "_pending" || !boundary.targetMmd) continue; // pending → ignore
nullByMmd.set(boundary.targetMmd, (nullByMmd.get(boundary.targetMmd) ?? 0) + 1);
}
// Trigger L2 for each MMD that reached threshold
for (const [mmdFile, count] of nullByMmd) {
if (count >= this.config.l2NullThreshold) {
// Threshold met: short delay (1s) so concurrent L1s merge into one L2
await this.deps.stateBackend.setTimerIfEarlier(
task.instanceId,
`offload-l2:${task.instanceId}:${sessionId}:${mmdFile}`,
Date.now() + 1_000,
);
this.deps.logger.info(`[offload-server] L2 timer set (fast, mmd=${mmdFile}, nullCount=${count})`);
} else if (count > 0) {
await this.deps.stateBackend.setTimerIfEarlier(
task.instanceId,
`offload-l2:${task.instanceId}:${sessionId}:${mmdFile}`,
Date.now() + 30_000,
);
this.deps.logger.info(`[offload-server] L2 timer set (mmd=${mmdFile}, nullCount=${count})`);
}
}
}
this.deps.logger.info(
`[offload-server] L1 complete: ${newEntries.length} entries produced (${Date.now() - startMs}ms)`,
);
}
// ═══════════════════════════════════════════════════════════════════════════
// L1.5: Task judgment — triggered by user new message
// ═══════════════════════════════════════════════════════════════════════════
async executeOffloadL15(task: TaskPayload, _signal?: AbortSignal): Promise<void> {
const startMs = Date.now();
const { sessionId, recentMessages, boundaryTimestamp } = task.data as {
sessionId: string;
recentMessages?: string;
boundaryTimestamp: string;
};
const storage = await this.resolveStorageOrThrow(task.instanceId);
const basePath = buildOffloadBasePath(sessionId);
// ─── Phase 1: Read-only snapshot + LLM call (NO LOCK) ─────────────────
// Multiple L1.5 tasks can execute this phase concurrently. Each operates
// on its own boundary and the LLM call (3-10s) does not block others.
// 1. Read current active MMD (snapshot for prompt building)
const preState = await this.readState(storage, basePath);
let currentMmd: { filename: string; content: string } | null = null;
if (preState.activeMmdFile) {
const content = await storage.readFile(`${basePath}/mmds/${preState.activeMmdFile}`);
if (content) {
currentMmd = { filename: preState.activeMmdFile, content };
}
}
// 2. List available MMDs → extract metas
const mmdsPrefix = `${basePath}/mmds/`;
const mmdFiles = await storage.readdirNames(mmdsPrefix, ".mmd");
const metas: MmdMeta[] = [];
for (const f of mmdFiles) {
const content = await storage.readFile(`${mmdsPrefix}${f}`);
if (content) {
metas.push(extractMmdMeta(f, content));
}
}
// 3. Build prompt & call LLM (no lock held — this is the expensive part)
const userPrompt = buildL15UserPrompt(
recentMessages ?? "",
currentMmd,
metas,
);
let judgment;
let rawResponse: string | undefined;
const l15LlmStart = Date.now();
try {
const response = await this.deps.llmClient.chat({
model: this.config.l15Model,
messages: [
{ role: "system", content: L15_SYSTEM_PROMPT },
{ role: "user", content: userPrompt },
],
temperature: this.config.l15Temperature,
max_tokens: this.config.l15MaxTokens,
timeoutMs: this.config.l15TimeoutMs,
});
rawResponse = response;
judgment = parseL15Response(response);
traceServerModelIo({
sessionId,
stage: "L1.5",
model: this.config.l15Model,
systemPrompt: L15_SYSTEM_PROMPT,
userPrompt,
responseContent: response,
status: "ok",
durationMs: Date.now() - l15LlmStart,
logger: this.deps.logger,
});
} catch (err) {
traceServerModelIo({
sessionId,
stage: "L1.5",
model: this.config.l15Model,
systemPrompt: L15_SYSTEM_PROMPT,
userPrompt,
responseContent: rawResponse ?? "",
status: "error",
errorMessage: String(err),
durationMs: Date.now() - l15LlmStart,
logger: this.deps.logger,
});
this.deps.logger.error(`[offload-server] L1.5 LLM failed:`, err);
throw err;
}
if (!judgment) {
this.deps.logger.warn(`[offload-server] L1.5: null response (parse failed), raw=${rawResponse?.slice(0, 500) ?? "(empty)"}`);
return;
}
this.deps.logger.info(
`[offload-server] L1.5: completed=${judgment.taskCompleted}, long=${judgment.isLongTask}, cont=${judgment.isContinuation}, label=${judgment.newTaskLabel ?? "none"} (LLM=${Date.now() - l15LlmStart}ms)`,
);
// Trace L1.5 decision
traceServerTaskDecision({
sessionId,
judgment: judgment as unknown as Record<string, unknown>,
durationMs: Date.now() - startMs,
logger: this.deps.logger,
});
// ─── Phase 2: Write phase (SHORT LOCK) ────────────────────────────────
// Only lock during state.json read-modify-write. Lock TTL is short (10s)
// because this phase only does file I/O (no LLM calls).
const lockKey = `offload-state:${task.instanceId}:${sessionId}`;
const lockOwner = task.id;
let locked = false;
for (let attempt = 0; attempt < 10; attempt++) {
locked = await this.deps.stateBackend.acquireLock(lockKey, lockOwner, 10_000);
if (locked) break;
await new Promise((r) => setTimeout(r, 100 + attempt * 50));
}
if (!locked) {
this.deps.logger.warn(`[offload-server] L1.5: failed to acquire write lock after 10 attempts (ts=${boundaryTimestamp}), skipping`);
return;
}
try {
// Re-read state (fresh, may have been modified by another L1.5 that finished earlier)
const state = await this.readState(storage, basePath);
// CAS check: find boundary by timestamp
const boundaryIdx = state.boundaries.findIndex((b) => b.timestamp === boundaryTimestamp);
if (boundaryIdx < 0) {
this.deps.logger.warn(`[offload-server] L1.5: boundary not found for ts=${boundaryTimestamp}, skipping`);
return;
}
// CAS check: already backfilled by another concurrent L1.5? Skip.
if (state.boundaries[boundaryIdx].targetMmd !== "_pending") {
this.deps.logger.info(`[offload-server] L1.5: boundary[ts=${boundaryTimestamp}] already backfilled, skipping`);
return;
}
// Apply task transition
await handleTaskTransition(state, judgment, storage, basePath);
// Backfill boundary targetMmd
state.boundaries[boundaryIdx].targetMmd = state.activeMmdFile;
// Update state
await this.writeState(storage, basePath, state);
// Check L2 trigger: scan ALL null entries grouped by targetMmd.
// This covers the case where L1 completed before L1.5 backfill —
// L1 skipped entries with _pending boundaries, so L1.5 must pick them up.
const entriesRaw = await storage.readFile(`${basePath}/entries.jsonl`);
if (entriesRaw) {
const allEntries = parseJsonl<OffloadEntry>(entriesRaw);
const l15NodeMapping = await this.readNodeMapping(storage, basePath);
// Group null entries by their resolved targetMmd (using all boundaries, not just current)
const nullByMmd = new Map<string, number>();
for (const e of allEntries) {
if (this.getEffectiveNodeId(e, l15NodeMapping) !== null || !e.timestamp) continue;
const boundary = this.findBoundaryByTimestamp(state.boundaries, e.timestamp);
if (!boundary) continue;
if (boundary.targetMmd === "_pending" || !boundary.targetMmd) continue;
nullByMmd.set(boundary.targetMmd, (nullByMmd.get(boundary.targetMmd) ?? 0) + 1);
}
for (const [mmdFile, nullCount] of nullByMmd) {
if (nullCount >= this.config.l2NullThreshold) {
await this.deps.stateBackend.setTimerIfEarlier(
task.instanceId,
`offload-l2:${task.instanceId}:${sessionId}:${mmdFile}`,
Date.now() + 1_000,
);
this.deps.logger.info(`[offload-server] L1.5: L2 timer set (fast, mmd=${mmdFile}, nullCount=${nullCount})`);
} else if (nullCount > 0) {
await this.deps.stateBackend.setTimerIfEarlier(
task.instanceId,
`offload-l2:${task.instanceId}:${sessionId}:${mmdFile}`,
Date.now() + 30_000,
);
this.deps.logger.info(`[offload-server] L1.5: L2 timer set after backfill (mmd=${mmdFile}, nullCount=${nullCount})`);
}
}
}
} finally {
if (locked) {
await this.deps.stateBackend.releaseLock(lockKey, lockOwner);
}
}
this.deps.logger.info(
`[offload-server] L1.5 complete: boundary[ts=${boundaryTimestamp}] (total=${Date.now() - startMs}ms)`,
);
}
// ═══════════════════════════════════════════════════════════════════════════
// L2: MMD generation / update + node_id backfill
// ═══════════════════════════════════════════════════════════════════════════
async executeOffloadL2(task: TaskPayload, _signal?: AbortSignal): Promise<void> {
const startMs = Date.now();
const sessionId = this.extractSessionId(task);
if (!sessionId) return;
let targetMmdFile: string | undefined;
const data = task.data as Record<string, unknown>;
targetMmdFile = data.targetMmdFile as string | undefined;
// Extract targetMmdFile from timer member if available
// Format: "offload-l2:{instanceId}:{sessionId}:{mmdFile}" → extract last segment ending in .mmd
if (!targetMmdFile && data.timerMember) {
const timerMember = data.timerMember as string;
const mmdMatch = timerMember.match(/(\d+-[^:]+\.mmd)$/);
if (mmdMatch) {
targetMmdFile = mmdMatch[1];
}
}
const storage = await this.resolveStorageOrThrow(task.instanceId);
const basePath = buildOffloadBasePath(sessionId);
const state = await this.readState(storage, basePath);
// Resolve targetMmdFile: from task data, timer member, or state (fallback)
if (!targetMmdFile) {
targetMmdFile = state.activeMmdFile ?? undefined;
}
if (!targetMmdFile) return;
// 1. Read all entries + node mapping
const entriesRaw = await storage.readFile(`${basePath}/entries.jsonl`);
if (!entriesRaw) return;
const allEntries = parseJsonl<OffloadEntry>(entriesRaw);
const nodeMapping = await this.readNodeMapping(storage, basePath);
// 2. Filter: entries belonging to targetMmdFile with null node_id (after join)
const relevantEntries = allEntries.filter((e) => {
if (this.getEffectiveNodeId(e, nodeMapping) !== null) return false;
if (!e.timestamp) return false;
const boundary = this.findBoundaryByTimestamp(state.boundaries, e.timestamp);
return boundary?.targetMmd === targetMmdFile;
});
if (relevantEntries.length === 0) return;
// Guard: if oldest null entry is older than 10 minutes, give up and assign fallback node_id
// to prevent infinite L2 retries when LLM consistently fails to map certain entries.
const L2_MAX_AGE_MS = 10 * 60 * 1000;
const oldestTs = relevantEntries.reduce((min, e) => {
const t = new Date(e.timestamp).getTime();
return t < min ? t : min;
}, Infinity);
if (Date.now() - oldestTs > L2_MAX_AGE_MS) {
this.deps.logger.warn(
`[offload-server] L2: ${relevantEntries.length} entries exceeded max age (10min), assigning fallback node_id`,
);
const fallbackMappings = relevantEntries.map((e) => ({
tool_call_id: e.tool_call_id,
node_id: `${targetMmdFile!.replace(/\.mmd$/, "")}-orphan`,
}));
const mappingPath = `${basePath}/node-mapping.jsonl`;
await storage.appendFile(mappingPath, serializeJsonl(fallbackMappings));
return;
}
// 3. Read existing MMD
const mmdPath = `${basePath}/mmds/${targetMmdFile}`;
const existingMmd = await storage.readFile(mmdPath);
// 4. Build prompt (include recent context for better MMD generation)
const recentHistory = await storage.readFile(`${basePath}/recent-context.txt`) ?? null;
const taskLabel = targetMmdFile.replace(/^\d+-/, "").replace(/\.mmd$/, "") || "task";
const prefixMatch = targetMmdFile.match(/^(\d+)-/);
const mmdPrefix = prefixMatch ? prefixMatch[1] : "000";
const charCount = existingMmd?.length ?? 0;
const userPrompt = buildL2UserPrompt({
existingMmd: existingMmd || null,
entries: relevantEntries,
recentHistory,
taskLabel,
mmdPrefix,
charCount,
});
// 5. Call LLM
let result: L2ParsedResponse | null;
let rawL2Response: string | undefined;
const l2LlmStart = Date.now();
try {
const response = await this.deps.llmClient.chat({
model: this.config.l2Model,
messages: [
{ role: "system", content: L2_SYSTEM_PROMPT },
{ role: "user", content: userPrompt },
],
temperature: this.config.l2Temperature,
max_tokens: this.config.l2MaxTokens,
timeoutMs: this.config.l2TimeoutMs,
});
rawL2Response = response;
result = parseL2Response(response);
traceServerModelIo({
sessionId: sessionId!,
stage: "L2",
model: this.config.l2Model,
systemPrompt: L2_SYSTEM_PROMPT,
userPrompt,
responseContent: response,
status: "ok",
durationMs: Date.now() - l2LlmStart,
logger: this.deps.logger,
});
} catch (err) {
traceServerModelIo({
sessionId: sessionId!,
stage: "L2",
model: this.config.l2Model,
systemPrompt: L2_SYSTEM_PROMPT,
userPrompt,
responseContent: rawL2Response ?? "",
status: "error",
errorMessage: String(err),
durationMs: Date.now() - l2LlmStart,
logger: this.deps.logger,
});
this.deps.logger.error(`[offload-server] L2 LLM failed:`, err);
throw err;
}
if (!result) {
this.deps.logger.warn(`[offload-server] L2: parse failed, raw=${rawL2Response?.slice(0, 500) ?? "(empty)"}`);
// Schedule retry so remaining null entries are not orphaned
await this.deps.stateBackend.setTimerIfEarlier(
task.instanceId,
`offload-l2:${task.instanceId}:${sessionId}:${targetMmdFile}`,
Date.now() + 30_000,
);
return;
}
// 6. Apply MMD update
const updatedMmd = this.applyL2Result(existingMmd ?? "", result);
await storage.writeFile(mmdPath, updatedMmd);
// 7. Backfill: write node mappings to separate file (avoids overwriting entries.jsonl)
const mappingEntries = Object.entries(result.nodeMapping).map(([toolCallId, nodeId]) => ({
tool_call_id: toolCallId,
node_id: nodeId,
}));
if (mappingEntries.length > 0) {
const mappingPath = `${basePath}/node-mapping.jsonl`;
await storage.appendFile(mappingPath, serializeJsonl(mappingEntries));
}
// 8. Check remaining null entries for this MMD — re-read node-mapping for accurate count
// (another L2 may have concurrently written mappings, or LLM may have missed some)
const freshNodeMapping = await this.readNodeMapping(storage, basePath);
const remainingNull = relevantEntries.filter(
(e) => !freshNodeMapping.has(e.tool_call_id) && !result!.nodeMapping[e.tool_call_id],
).length;
if (remainingNull > 0) {
await this.deps.stateBackend.setTimerIfEarlier(
task.instanceId,
`offload-l2:${task.instanceId}:${sessionId}:${targetMmdFile}`,
Date.now() + 30_000,
);
this.deps.logger.info(`[offload-server] L2 retry timer set (mmd=${targetMmdFile}, remainingNull=${remainingNull})`);
}
this.deps.logger.info(
`[offload-server] L2 complete: ${Object.keys(result.nodeMapping).length} entries mapped, action=${result.fileAction} (${Date.now() - startMs}ms)`,
);
}
// ═══════════════════════════════════════════════════════════════════════════
// Helpers
// ═══════════════════════════════════════════════════════════════════════════
/**
* Extract sessionId from task data or task.sessionId.
*/
private extractSessionId(task: TaskPayload): string | undefined {
const data = task.data as Record<string, unknown> | undefined;
// Prefer explicit sessionId in data
if (data?.sessionId && typeof data.sessionId === "string") {
return data.sessionId;
}
// Fallback: task-level sessionId
if (task.sessionId) {
return task.sessionId;
}
return undefined;
}
private async resolveStorageOrThrow(instanceId: string): Promise<StorageAdapter> {
const storage = await this.deps.resolveStorage(instanceId);
if (!storage) throw new Error(`Storage unavailable for instance ${instanceId}`);
return storage;
}
private async readState(storage: StorageAdapter, basePath: string): Promise<OffloadState> {
const raw = await storage.readFile(`${basePath}/state.json`);
if (!raw) return defaultOffloadState();
try {
return { ...defaultOffloadState(), ...JSON.parse(raw) };
} catch {
return defaultOffloadState();
}
}
private async writeState(storage: StorageAdapter, basePath: string, state: OffloadState): Promise<void> {
await storage.writeFile(`${basePath}/state.json`, JSON.stringify(state));
}
private findBoundaryByTimestamp(
boundaries: OffloadState["boundaries"],
entryTimestamp: string,
): OffloadState["boundaries"][number] | null {
if (boundaries.length === 0) return null;
// Find the last boundary whose timestamp <= entryTimestamp
let result: OffloadState["boundaries"][number] | null = null;
for (const b of boundaries) {
if (b.timestamp <= entryTimestamp) result = b;
else break;
}
return result;
}
private applyL2Result(existingMmd: string, result: L2ParsedResponse): string {
if (result.fileAction === "write" && result.mmdContent) {
return result.mmdContent;
}
if (result.fileAction === "replace" && result.replaceBlocks?.length) {
const lines = existingMmd.split("\n");
// Sort blocks by startLine descending to avoid offset issues
const sorted = [...result.replaceBlocks]
.filter((block) => block.startLine >= 1 && block.endLine >= block.startLine && block.startLine <= lines.length)
.sort((a, b) => b.startLine - a.startLine);
for (const block of sorted) {
const start = block.startLine - 1; // 0-based
const end = Math.min(block.endLine, lines.length); // clamp endLine
const deleteCount = end - block.startLine + 1;
const newLines = block.content.split("\n");
lines.splice(start, deleteCount, ...newLines);
}
return lines.join("\n");
}
return existingMmd;
}
/**
* Read node-mapping.jsonl and build a tool_call_id node_id map.
*/
private async readNodeMapping(storage: StorageAdapter, basePath: string): Promise<Map<string, string>> {
const raw = await storage.readFile(`${basePath}/node-mapping.jsonl`);
const map = new Map<string, string>();
if (!raw) return map;
const lines = parseJsonl<{ tool_call_id: string; node_id: string }>(raw);
for (const line of lines) {
if (line.tool_call_id && line.node_id) {
map.set(line.tool_call_id, line.node_id);
}
}
return map;
}
/**
* Get effective node_id for an entry: check node-mapping first, then entry's own field.
*/
private getEffectiveNodeId(entry: OffloadEntry, nodeMapping: Map<string, string>): string | null {
return nodeMapping.get(entry.tool_call_id) ?? entry.node_id;
}
}
+336
View File
@@ -0,0 +1,336 @@
/**
* Opik observability tracer for offload server.
* Wraps the opik npm package with graceful degradation when not installed.
* Configuration is read from environment variables (no OpenClaw plugin config).
*/
// Opik client types (minimal shape to avoid hard dependency)
interface OpikClient {
trace(params: Record<string, unknown>): OpikTrace;
flush(): Promise<void>;
}
interface OpikTrace {
update(params: Record<string, unknown>): void;
end(): void;
span(params: Record<string, unknown>): OpikSpan;
}
interface OpikSpan {
update(params: Record<string, unknown>): void;
end(): void;
}
// ─── Module State ────────────────────────────────────────────────────────────
let client: OpikClient | null = null;
let tracerEnabled = false;
let tracerInitTried = false;
// ─── Helpers ─────────────────────────────────────────────────────────────────
function extractLayerTag(stage: string): string {
const match = stage.match(/^(L\d+(?:\.\d+)?)/i);
if (!match) return "Lx-unknown";
return match[1].toUpperCase();
}
function durationBucketTag(ms: number): string {
if (typeof ms !== "number" || ms < 0) return "duration:unknown";
if (ms < 1000) return "duration:<1s";
if (ms < 5000) return "duration:1-5s";
if (ms < 15000) return "duration:5-15s";
if (ms < 30000) return "duration:15-30s";
return "duration:>30s";
}
function formatDuration(ms: number): string {
if (typeof ms !== "number" || ms < 0) return "?";
if (ms < 1000) return `${Math.round(ms)}ms`;
return `${(ms / 1000).toFixed(2)}s`;
}
// ─── Logger Interface ────────────────────────────────────────────────────────
export interface TracerLogger {
info: (message: string, ...args: unknown[]) => void;
warn: (message: string, ...args: unknown[]) => void;
debug?: (message: string, ...args: unknown[]) => void;
}
// ─── Init ────────────────────────────────────────────────────────────────────
/**
* Initialize the offload server Opik tracer.
* Reads config from environment variables:
* OPIK_ENABLED set to "true" to enable (default: disabled)
* OPIK_URL_OVERRIDE Opik server URL
* OPIK_API_KEY API key
* OPIK_WORKSPACE workspace name (default: "default")
* OPIK_PROJECT_NAME project name (default: "openclaw-offload-server")
*/
export async function initServerOpikTracer(logger: TracerLogger): Promise<void> {
if (tracerInitTried) return;
tracerInitTried = true;
try {
const enabled = process.env.OPIK_ENABLED === "true";
if (!enabled) {
logger.debug?.("[offload-server] Opik tracer disabled (OPIK_ENABLED != true)");
return;
}
const apiUrl = process.env.OPIK_URL_OVERRIDE;
const apiKey = process.env.OPIK_API_KEY;
const workspaceName = process.env.OPIK_WORKSPACE ?? "default";
const projectName = process.env.OPIK_PROJECT_NAME ?? "openclaw-offload-server";
// Dynamic import — graceful when opik is not installed
let OpikConstructor: new (params: Record<string, unknown>) => OpikClient;
let disableOpikLogger: (() => void) | undefined;
try {
const opikModule = await import("opik") as {
Opik: new (params: Record<string, unknown>) => OpikClient;
disableLogger?: () => void;
setLoggerLevel?: (level: string) => void;
};
OpikConstructor = opikModule.Opik;
disableOpikLogger = opikModule.disableLogger;
} catch {
logger.debug?.("[offload-server] opik package not available, tracer disabled");
return;
}
// Suppress opik internal logs (flush messages, ANSI color noise)
if (disableOpikLogger) {
disableOpikLogger();
}
client = new OpikConstructor({
...(apiKey ? { apiKey } : {}),
...(apiUrl ? { apiUrl } : {}),
workspaceName,
projectName,
});
tracerEnabled = true;
logger.info(
`[offload-server] Opik tracer enabled: project=${projectName}, workspace=${workspaceName}`,
);
} catch (err) {
tracerEnabled = false;
client = null;
logger.warn(`[offload-server] Opik tracer init failed: ${String(err)}`);
}
}
/**
* Check if the tracer is enabled and ready to trace.
*/
export function isTracerEnabled(): boolean {
return tracerEnabled && client !== null;
}
// ─── Trace: Model I/O (L1/L1.5/L2 LLM calls) ───────────────────────────────
/**
* Trace LLM model I/O for offload server L1/L1.5/L2 stages.
*/
export function traceServerModelIo(params: {
sessionId: string;
stage: string;
model: string;
systemPrompt: string;
userPrompt: string;
responseContent: string;
usage?: { promptTokens?: number; completionTokens?: number; totalTokens?: number };
status: "ok" | "error";
errorMessage?: string;
durationMs: number;
logger?: TracerLogger;
}): void {
if (!tracerEnabled || !client) return;
try {
const layerTag = extractLayerTag(params.stage);
const threadId = params.sessionId || `offload-server-${Date.now()}`;
const dur = params.durationMs;
const durStr = formatDuration(dur);
const durBucket = durationBucketTag(dur);
const skTag = `session:${params.sessionId || "unknown"}`;
const trace = client.trace({
name: `${params.model} · offload-server · ${params.stage} · ${durStr}`,
threadId,
metadata: {
plugin: "openclaw-offload-server",
category: "llm",
stage: params.stage,
layer: layerTag,
model: params.model,
sessionId: params.sessionId,
durationMs: dur,
duration: durStr,
},
tags: ["offload-server", "llm", layerTag, durBucket, skTag],
});
const span = trace.span({
name: `${params.model} · ${params.stage} · ${durStr}`,
type: "llm",
model: params.model,
input: {
systemPrompt: params.systemPrompt,
userPrompt: params.userPrompt,
},
metadata: {
stage: params.stage,
layer: layerTag,
sessionId: params.sessionId,
durationMs: dur,
duration: durStr,
},
});
span.update({
output: {
responseContent: params.responseContent,
usage: params.usage,
durationMs: dur,
duration: durStr,
error: params.errorMessage,
},
metadata: {
status: params.status,
durationMs: dur,
},
});
span.end();
trace.end();
void client.flush().catch(() => undefined);
} catch (err) {
params.logger?.warn?.(`[offload-server] Opik model I/O trace failed: ${String(err)}`);
}
}
// ─── Trace: Compaction Decision ──────────────────────────────────────────────
/**
* Trace compaction (L3) decision and results.
*/
export function traceServerCompaction(params: {
sessionId: string;
level: string;
ratio: number;
contextWindow: number;
totalTokensBefore: number;
totalTokensAfter: number;
originalMsgCount: number;
compactedMsgCount: number;
report: Record<string, unknown>;
messages: unknown[];
durationMs: number;
logger?: TracerLogger;
}): void {
if (!tracerEnabled || !client) return;
try {
const threadId = params.sessionId || `offload-server-${Date.now()}`;
const dur = params.durationMs;
const durStr = formatDuration(dur);
const durBucket = durationBucketTag(dur);
const skTag = `session:${params.sessionId || "unknown"}`;
const trace = client.trace({
name: `compaction · L3 · ${params.level} · ${durStr} [${params.sessionId}]`,
threadId,
input: {
level: params.level,
ratio: params.ratio,
contextWindow: params.contextWindow,
totalTokensBefore: params.totalTokensBefore,
originalMsgCount: params.originalMsgCount,
},
metadata: {
plugin: "openclaw-offload-server",
category: "compaction",
stage: "L3",
layer: "L3",
level: params.level,
sessionId: params.sessionId,
durationMs: dur,
duration: durStr,
},
tags: ["offload-server", "compaction", "L3", `level:${params.level}`, durBucket, skTag],
});
// Serialize messages for full snapshot
const serializedMessages = params.messages.map((msg: any, i: number) => {
const role = msg.role ?? "unknown";
const content = typeof msg.content === "string"
? msg.content
: Array.isArray(msg.content)
? msg.content.map((c: any) => {
if (c.type === "text") return c.text;
if (c.type === "tool_use") return `[tool_use: ${c.name} id=${c.id}]`;
if (c.type === "tool_result") return `[tool_result: id=${c.tool_use_id} content=${typeof c.content === "string" ? c.content.slice(0, 500) : JSON.stringify(c.content).slice(0, 500)}]`;
return `[${c.type ?? "unknown"}]`;
}).join("\n")
: "";
return { i, role, content, ...(msg._mmdContextMessage ? { mmdCtx: true } : {}), ...(msg._offloaded ? { offloaded: true } : {}) };
});
trace.update({
output: {
totalTokensAfter: params.totalTokensAfter,
compactedMsgCount: params.compactedMsgCount,
tokenReduction: params.totalTokensBefore - params.totalTokensAfter,
report: params.report,
messages: serializedMessages,
},
});
trace.end();
void client.flush().catch(() => undefined);
} catch (err) {
params.logger?.warn?.(`[offload-server] Opik compaction trace failed: ${String(err)}`);
}
}
// ─── Trace: Task Decision (L1.5 judgment result) ─────────────────────────────
/**
* Trace L1.5 task judgment decision.
*/
export function traceServerTaskDecision(params: {
sessionId: string;
judgment: Record<string, unknown>;
durationMs: number;
logger?: TracerLogger;
}): void {
if (!tracerEnabled || !client) return;
try {
const threadId = params.sessionId || `offload-server-${Date.now()}`;
const dur = params.durationMs;
const durStr = formatDuration(dur);
const skTag = `session:${params.sessionId || "unknown"}`;
const trace = client.trace({
name: `task-decision · L1.5 · ${durStr} [${params.sessionId}]`,
threadId,
input: {
stage: "L1.5",
sessionId: params.sessionId,
},
metadata: {
plugin: "openclaw-offload-server",
category: "decision",
stage: "L1.5",
layer: "L1.5",
sessionId: params.sessionId,
durationMs: dur,
duration: durStr,
},
tags: ["offload-server", "decision", "L1.5", skTag],
});
trace.update({ output: params.judgment });
trace.end();
void client.flush().catch(() => undefined);
} catch (err) {
params.logger?.warn?.(`[offload-server] Opik task decision trace failed: ${String(err)}`);
}
}
+92
View File
@@ -0,0 +1,92 @@
/**
* JSON / Mermaid extraction utilities for LLM response parsing.
*/
/**
* Extract JSON from raw LLM output. Tolerates markdown fences, extra text.
*/
export function extractJson<T>(raw: string): T | null {
if (!raw || typeof raw !== "string") return null;
const trimmed = raw.trim();
// 1. Direct parse
try {
return JSON.parse(trimmed) as T;
} catch {
// continue
}
// 2. Extract from ```json ... ``` fence
const jsonFenceMatch = trimmed.match(/```(?:json)?\s*\n?([\s\S]*?)```/);
if (jsonFenceMatch) {
try {
return JSON.parse(jsonFenceMatch[1].trim()) as T;
} catch {
// continue
}
}
// 3. Extract first { ... } (greedy last })
const objStart = trimmed.indexOf("{");
const objEnd = trimmed.lastIndexOf("}");
if (objStart !== -1 && objEnd > objStart) {
try {
return JSON.parse(trimmed.slice(objStart, objEnd + 1)) as T;
} catch {
// continue
}
}
// 4. Extract first [ ... ] (greedy last ])
const arrStart = trimmed.indexOf("[");
const arrEnd = trimmed.lastIndexOf("]");
if (arrStart !== -1 && arrEnd > arrStart) {
try {
return JSON.parse(trimmed.slice(arrStart, arrEnd + 1)) as T;
} catch {
// continue
}
}
return null;
}
/**
* Extract mermaid content from a ```mermaid ... ``` code fence.
*/
export function extractMermaidFromFence(raw: string): string | null {
if (!raw) return null;
const match = raw.match(/```mermaid\s*\n?([\s\S]*?)```/);
return match ? match[1].trim() : null;
}
/**
* Parse JSONL (newline-delimited JSON) string into array.
* Corrupted lines are silently skipped (logged via optional callback).
*/
export function parseJsonl<T>(
content: string,
onBadLine?: (line: string, error: unknown) => void,
): T[] {
if (!content || !content.trim()) return [];
const results: T[] = [];
for (const line of content.split("\n")) {
const trimmed = line.trim();
if (trimmed.length === 0) continue;
try {
results.push(JSON.parse(trimmed) as T);
} catch (err) {
onBadLine?.(trimmed, err);
}
}
return results;
}
/**
* Serialize array to JSONL string (trailing newline).
*/
export function serializeJsonl<T>(items: T[]): string {
if (items.length === 0) return "";
return items.map((item) => JSON.stringify(item)).join("\n") + "\n";
}
+41
View File
@@ -0,0 +1,41 @@
/**
* L1 Response Parser extracts OffloadEntry[] from LLM output.
*/
import { extractJson } from "./json-utils.js";
import type { OffloadEntry } from "../types.js";
interface RawL1Entry {
tool_call?: string;
summary?: string;
tool_call_id?: string;
timestamp?: string;
score?: number;
}
/**
* Parse L1 LLM response into OffloadEntry array.
* Tolerant of markdown wrapping, missing fields, etc.
*/
export function parseL1Response(raw: string): OffloadEntry[] {
const parsed = extractJson<RawL1Entry[]>(raw);
if (!parsed || !Array.isArray(parsed)) return [];
const entries: OffloadEntry[] = [];
for (const item of parsed) {
if (!item || typeof item !== "object") continue;
const toolCallId = item.tool_call_id ?? "";
if (!toolCallId) continue;
entries.push({
tool_call_id: toolCallId,
tool_call: item.tool_call ?? "",
summary: item.summary ?? "",
timestamp: item.timestamp ?? "",
score: typeof item.score === "number" ? Math.min(10, Math.max(0, item.score)) : 5,
node_id: null,
seq: -1,
});
}
return entries;
}
+57
View File
@@ -0,0 +1,57 @@
/**
* L1.5 Response Parser extracts TaskJudgment from LLM output.
*/
import { extractJson } from "./json-utils.js";
import type { TaskJudgment } from "../types.js";
interface RawL15Response {
taskCompleted?: boolean | null;
isContinuation?: boolean | null;
isLongTask?: boolean | null;
continuationMmdFile?: string | null;
newTaskLabel?: string | null;
}
/**
* Parse L1.5 LLM response into TaskJudgment.
* Returns null if completely unparseable or all-null (LLM unavailable).
*/
export function parseL15Response(raw: string): TaskJudgment | null {
const parsed = extractJson<RawL15Response>(raw);
if (!parsed || typeof parsed !== "object") return null;
if (
parsed.taskCompleted == null &&
parsed.isContinuation == null &&
parsed.isLongTask == null
) {
return null;
}
return {
taskCompleted: toBool(parsed.taskCompleted),
isContinuation: toBool(parsed.isContinuation),
isLongTask: toBool(parsed.isLongTask),
continuationMmdFile:
typeof parsed.continuationMmdFile === "string" && isSafeFilename(parsed.continuationMmdFile)
? parsed.continuationMmdFile
: undefined,
newTaskLabel:
typeof parsed.newTaskLabel === "string" ? parsed.newTaskLabel : undefined,
};
}
/** Safely coerce LLM value to boolean, handling string "false"/"0". */
function toBool(value: unknown): boolean {
if (typeof value === "string") {
return value.toLowerCase() !== "false" && value !== "0" && value !== "";
}
return Boolean(value);
}
/** Validate that a filename is safe (no path traversal or special chars). */
function isSafeFilename(name: string): boolean {
if (!name) return false;
if (name.includes("/") || name.includes("\\") || name.includes("..")) return false;
return /^[a-zA-Z0-9_.\-]+$/.test(name);
}
+76
View File
@@ -0,0 +1,76 @@
/**
* L2 Response Parser extracts MMD generation results from LLM output.
*/
import { extractJson, extractMermaidFromFence } from "./json-utils.js";
import type { L2ParsedResponse } from "../types.js";
interface RawL2Response {
file_action?: string;
mmd_content?: string | null;
replace_blocks?: Array<{
start_line?: number | string;
end_line?: number | string;
content?: string;
}> | null;
node_mapping?: Record<string, string>;
}
/**
* Parse L2 LLM response into structured result.
* Returns null if parsing fails completely.
*/
export function parseL2Response(raw: string): L2ParsedResponse | null {
const parsed = extractJson<RawL2Response>(raw);
if (!parsed || typeof parsed !== "object") {
// Fallback: try extracting ```mermaid ... ``` code block
const mmd = extractMermaidFromFence(raw);
if (mmd) {
return { fileAction: "write", mmdContent: mmd, nodeMapping: {} };
}
return null;
}
const fileAction = parsed.file_action === "replace" ? "replace" : "write";
// Extract mmd_content (may be wrapped in code fence)
let mmdContent: string | undefined;
if (fileAction === "write") {
if (parsed.mmd_content) {
mmdContent =
extractMermaidFromFence(parsed.mmd_content) ?? parsed.mmd_content;
} else {
const fallbackMmd = extractMermaidFromFence(raw);
if (fallbackMmd) mmdContent = fallbackMmd;
}
}
// Parse replace_blocks
let replaceBlocks: L2ParsedResponse["replaceBlocks"] | undefined;
if (fileAction === "replace" && Array.isArray(parsed.replace_blocks)) {
replaceBlocks = [];
for (const block of parsed.replace_blocks) {
if (!block || typeof block !== "object") continue;
const startLine = Number(block.start_line);
const endLine = Number(block.end_line);
if (isNaN(startLine) || isNaN(endLine)) continue;
let content = block.content ?? "";
const extracted = extractMermaidFromFence(content);
if (extracted) content = extracted;
replaceBlocks.push({ startLine, endLine, content });
}
}
// Parse node_mapping
const nodeMapping: Record<string, string> = {};
if (parsed.node_mapping && typeof parsed.node_mapping === "object") {
for (const [key, value] of Object.entries(parsed.node_mapping)) {
if (typeof value === "string") {
nodeMapping[key] = value;
}
}
}
return { fileAction, mmdContent, replaceBlocks, nodeMapping };
}
+81
View File
@@ -0,0 +1,81 @@
/**
* L1 Summarization Prompt converts ToolPairs into OffloadEntry summaries.
*/
import type { ToolPair } from "../types.js";
const PARAMS_MAX_LEN = 500;
const RESULT_MAX_LEN = 2000;
const COMPRESS_THRESHOLD = 200;
export const L1_SYSTEM_PROMPT = `你是一个专为 AI 编码助手提供支持的"工具结果摘要器"。你的核心任务是深度理解当前的对话上下文,并将繁杂的工具调用与执行结果(一对toolcall和tool result整合成一条summary输出),提炼为高信息密度的 JSON 数组。
1.
2. "发现了什么关键线索""做了什么关键动作""修改了什么具体内容""遇到了什么具体报错"
3.
JSON [{...}]****
- "tool_call":
· tool pair [NEEDS_COMPRESS]+150/
exec({"command":"python3 -c 'import csv; ...200行脚本...'"}) "exec: 运行 Python xx/xx/xx.sh,标明具体路径和文件)脚本分析 sales_channels.csv 数据质量"
write_file({"path":"/root/app.py","content":"...5000字符..."}) "write_file: 写入 /root/app.py (Flask 应用主文件),大致内容是……"
· [NEEDS_COMPRESS]
- "summary": 200/
- "tool_call_id": tool_call_id
- "timestamp": ISO 8601
- "score"****: summary对于原文的可替代性0-1010summary越能替代原文
JSON `;
/**
* Build the L1 user prompt for summarization.
*/
export function buildL1UserPrompt(
recentContext: string,
pairs: ToolPair[],
): string {
const parts: string[] = [];
parts.push("## 最近的对话上下文(用于理解当前任务):");
parts.push(recentContext || "(无可用上下文)");
parts.push("\n## Tool call/result pairs to summarize:");
for (let i = 0; i < pairs.length; i++) {
const p = pairs[i];
const paramsStr = truncate(stringify(p.params), PARAMS_MAX_LEN);
const resultStr = truncate(stringify(p.result), RESULT_MAX_LEN);
const canonical = `${p.toolName}(${stringify(p.params)})`;
const needsCompress = canonical.length > COMPRESS_THRESHOLD;
parts.push(`--- Tool Pair ${i + 1} ---`);
parts.push(`tool_call_id: ${p.toolCallId}`);
parts.push(`timestamp: ${p.timestamp}`);
if (needsCompress) {
parts.push(`Tool: ${p.toolName} [NEEDS_COMPRESS]`);
} else {
parts.push(`Tool: ${p.toolName}`);
}
parts.push(`Params: ${paramsStr}`);
parts.push(`Result: ${resultStr}\n`);
}
parts.push("Summarize each pair into the JSON array format described.");
return parts.join("\n");
}
function stringify(value: unknown): string {
if (value == null) return "";
if (typeof value === "string") return value;
try {
return JSON.stringify(value);
} catch {
return String(value);
}
}
function truncate(s: string, maxLen: number): string {
if (s.length <= maxLen) return s;
return s.slice(0, maxLen) + "...";
}
+81
View File
@@ -0,0 +1,81 @@
/**
* L1.5 Task Judgment Prompt determines task lifecycle.
*/
import type { MmdMeta } from "../types.js";
export const L15_SYSTEM_PROMPT = `你是一个面向 AI 编码助手的"任务生命周期门神"。
JSON
1. - recentMessages"继续排查""宣布完工(如:跑通了)""单轮闲聊问答""开启全新需求"
2. - currentMmd线 currentMmd Mermaid taskGoal statusdone/doing/todo summary done taskCompleted true doing bug false(currentMmd)
3. - availableMmdsisLongTask=true taskCompleted=true/ availableMmds taskGoal isContinuation=true
JSON
JSON
{
"taskCompleted": boolean, // 当前任务是否已结束(如果 currentMmd 为 none,这里必须填 true
"isLongTask": boolean, // 最新诉求是否是需要多步操作的复杂工程(普通技术问答、闲聊填 false)
"isContinuation": boolean, // 是否在延续 availableMmds 中的历史任务
"continuationMmdFile": "string|null", // 若延续旧任务,精确填入 availableMmds 中的文件名(不含路径前缀),否则为 null
"newTaskLabel": "string|null" // 若是全新长任务,生成简短标签(≤30字符,kebab-case,如 "refactor-api"),否则为 null
}
JSON `;
export interface L15CurrentMmd {
filename: string;
content: string;
}
/**
* Build the L1.5 user prompt for task judgment.
*/
export function buildL15UserPrompt(
recentMessages: string,
currentMmd: L15CurrentMmd | null,
metas: MmdMeta[],
): string {
const parts: string[] = [];
parts.push("## 1. 最近的对话上下文 (Recent messages):");
parts.push(recentMessages);
parts.push("\n## 2. 当前挂载的任务图 (Active Mermaid — 完整内容):");
if (currentMmd && currentMmd.filename) {
parts.push(`**File:** ${currentMmd.filename}`);
parts.push(`\n\`\`\`mermaid\n${currentMmd.content}\n\`\`\``);
} else {
parts.push("(none - 当前处于闲置状态,无活跃任务)");
}
parts.push("\n## 3. 历史可用的任务图 (Available Mermaid task files):");
if (metas.length === 0) {
parts.push("(none - 暂无历史长任务)");
} else {
for (const m of metas) {
const total = m.doneCount + m.doingCount + m.todoCount;
parts.push(`- **${m.filename}**`);
parts.push(` taskGoal: ${m.taskGoal}`);
parts.push(
` progress: ${m.doneCount}/${total} done, ${m.doingCount} doing, ${m.todoCount} todo`,
);
if (m.updatedTime) {
parts.push(` lastUpdated: ${m.updatedTime}`);
}
if (m.nodeSummaries && m.nodeSummaries.length > 0) {
parts.push(" recentNodes:");
for (const n of m.nodeSummaries) {
parts.push(` - [${n.nodeId}] (${n.status}) ${n.summary}`);
}
}
parts.push("");
}
}
parts.push(
"请严格根据系统指令的【三步思考链路】进行研判,并输出合法的 JSON 对象。",
);
return parts.join("\n");
}
+113
View File
@@ -0,0 +1,113 @@
/**
* L2 MMD Generation Prompt generates/updates Mermaid flowcharts.
*/
import type { OffloadEntry } from "../types.js";
export const L2_SYSTEM_PROMPT = `你是一个究极实用主义的 AI 任务拓扑架构师与视觉叙事者。
LLM模型能看懂 Mermaid (flowchart TD) "过去""未来""雷区"
1.
2. ()status: blockedfail信息则不需要记录
3. summary150"得出了什么结论""发生了什么实质改变"
4. node_id
Token "认知锚点"使mmd形状来代表不同的节点逻辑
1. "领域" summary 150"发现死锁""依赖冲突""已修复"
2. 使线-->||线-.->||"依赖树""假设验证环"
3. (Token )
- replace ()
- write ()
Existing Mermaid content "L1: ..." replace MMD
线
1.NodeID["阶段名: 宏观动作简述<br/>status: done|doing|paused|blocked <br/>summary: 核心结论摘要<br/>Timestamp: ISO8601"]
2. 宿 tool_call_id node_mapping Node IDMMD里的每一个node都应该有源头的tool_call消息来源Node_id和tool_call_id是一对多的关系
3. mmd文件大小控制在4000字以内
1. %%{ "taskGoal": "一句话总结此次任务的目标(可动态更新)", "progress0-100": "进度百分比(严格点,几乎确认完成再打到90+)", createdTime": "ISO时间", "updatedTime": "ISO时间" }%%updatedTime为node中的最新时间
2. Timestamp ISO
JSON
Mermaid mmd_content replace_blocks content \`\`\`mermaid ... \`\`\` 代码块包裹起来。必须输出如下 JSON 结构:
{
"file_action": "replace 或 write",
"mmd_content": "完整的、带转义的 .mmd 代码,必须用 \`\`\`mermaid ... \`\`\` 包裹。(仅在 file_action 为 write 时填写,否则必须设为 null",
"replace_blocks": [
{
"start_line": "需要更新范围的起始行号(整数,对应 Existing Mermaid content 中的 L 标号)",
"end_line": "需要更新范围的结束行号(整数,包含该行)。要在某行之前插入新内容而不删除任何行,将 start_line 设为该行号,end_line 设为 start_line - 1",
"content": "替换后的新内容(不需要带行号前缀),必须用 \`\`\`mermaid ... \`\`\` 包裹"
}
],
"node_mapping": {
"tool_call_id_1": "001-N1",
"tool_call_id_2": "001-N1"
}
}
node_mapping Node ID MMD 使 ID MMD prefix "001-N1" ID "N1"
JSON `;
/**
* Build the L2 user prompt for MMD generation.
*/
export function buildL2UserPrompt(opts: {
existingMmd: string | null;
entries: OffloadEntry[];
recentHistory?: string | null;
currentTurn?: string | null;
taskLabel: string;
mmdPrefix: string;
charCount: number;
}): string {
const { existingMmd, entries, recentHistory, currentTurn, taskLabel, mmdPrefix, charCount } = opts;
const parts: string[] = [];
// History section
if (recentHistory) {
parts.push(`## 近期对话历史:\n${recentHistory}`);
} else {
parts.push("## 近期对话历史:\n(无可用历史)");
}
if (currentTurn) {
parts.push(`\n## 当前最新一轮:\n${currentTurn}`);
}
parts.push(`\n## MMD prefix: ${mmdPrefix}`);
parts.push(`(所有节点 ID 必须以此前缀开头,如 ${mmdPrefix}-N1, ${mmdPrefix}-N2...`);
parts.push(`\n## Current task label: ${taskLabel}`);
// Char count warning
if (charCount > 2500) {
parts.push(`\n## Current MMD size: ${charCount} chars (budget: 4000 chars)`);
parts.push("⚠ 接近上限,请积极合并节点、精简 summary,优先使用 replace 模式微调而非 write 全量重写。");
} else if (charCount > 2000) {
parts.push(`\n## Current MMD size: ${charCount} chars (budget: 4000 chars)`);
parts.push("注意控制增长,合并同类节点。");
}
// Existing MMD with line numbers
parts.push("\n## Existing Mermaid content:");
if (existingMmd) {
const lines = existingMmd.split("\n");
for (let i = 0; i < lines.length; i++) {
parts.push(`L${i + 1}: ${lines[i]}`);
}
} else {
parts.push("(empty — create new)");
}
// New entries
parts.push("\n## New offload entries to incorporate:");
for (let i = 0; i < entries.length; i++) {
const e = entries[i];
parts.push(`${i + 1}. [${e.tool_call_id}] ${e.tool_call}${e.summary} (${e.timestamp})`);
}
parts.push("\n请根据系统指令生成/更新 Mermaid 流程图,并输出合法的 JSON 对象(含 node_mapping)。");
return parts.join("\n");
}
+88
View File
@@ -0,0 +1,88 @@
/**
* Offload V2 Router route registration and dispatch.
*/
import type http from "node:http";
import type { StorageAdapter } from "../core/storage/adapter.js";
import type { IStateBackend } from "../core/state/types.js";
import type { OffloadExecutorConfig } from "./types.js";
import { defaultOffloadConfig } from "./types.js";
import { parseV2Auth, successEnvelope, errorEnvelope, makeRequestId } from "../gateway/v2-router.js";
import { handleIngest } from "./ingest-handler.js";
import { handleMmdQuery } from "./mmd-handler.js";
import { handleCompaction } from "./compact/compaction-handler.js";
import { MmdQuerySchema } from "./schemas.js";
export interface OffloadV2Deps {
resolveStorage?: (instanceId: string) => Promise<StorageAdapter | undefined>;
getStorage: () => StorageAdapter | undefined;
logger: { info: (...args: unknown[]) => void; warn: (...args: unknown[]) => void; error: (...args: unknown[]) => void };
stateBackend?: IStateBackend;
config?: OffloadExecutorConfig;
}
/**
* Handle offload V2 routes. Returns true if the request was handled.
*/
export async function handleOffloadV2Route(
req: http.IncomingMessage,
res: http.ServerResponse,
pathname: string,
method: string,
parseJsonBody: <T>(req: http.IncomingMessage) => Promise<T>,
sendJson: (res: http.ServerResponse, status: number, body: unknown) => void,
deps: OffloadV2Deps,
): Promise<boolean> {
if (!pathname.startsWith("/v2/offload/")) return false;
const requestId = makeRequestId();
// Auth
const auth = parseV2Auth(req, res, requestId, sendJson);
if (!auth) return true; // 401 already sent
// Resolve storage
const storage =
(await deps.resolveStorage?.(auth.serviceId)) ?? deps.getStorage();
if (!storage) {
sendJson(res, 503, errorEnvelope(503, "Storage unavailable", requestId));
return true;
}
const config = deps.config ?? defaultOffloadConfig();
// Normalize trailing slash for consistent route matching
const normalizedPath = pathname.endsWith("/") && pathname.length > 1 ? pathname.slice(0, -1) : pathname;
const route = `${method} ${normalizedPath}`;
switch (route) {
case "POST /v2/offload/ingest":
await handleIngest(req, res, auth, {
storage,
stateBackend: deps.stateBackend,
config,
logger: deps.logger,
}, requestId, parseJsonBody, sendJson, successEnvelope, errorEnvelope);
return true;
case "POST /v2/offload/query-mmd": {
const body = await parseJsonBody<{ session_id?: string; limit?: number }>(req);
const parsed = MmdQuerySchema.safeParse(body);
if (!parsed.success) {
sendJson(res, 400, errorEnvelope(400, "missing or invalid session_id in body", requestId));
return true;
}
await handleMmdQuery(req, res, auth, storage, requestId, sendJson, successEnvelope, errorEnvelope, parsed.data.session_id, parsed.data.limit);
return true;
}
case "POST /v2/offload/compact":
await handleCompaction(req, res, auth, {
storage,
config,
logger: deps.logger,
}, requestId, parseJsonBody, sendJson, successEnvelope, errorEnvelope);
return true;
default:
return false;
}
}
+80
View File
@@ -0,0 +1,80 @@
/**
* Offload Server Request validation schemas (Zod).
*/
import { z } from "zod";
/** Safe session ID: alphanumeric, underscore, hyphen, dot, colon allowed. No slashes or path traversal. Max 500 chars. */
const safeSessionId = z.string().min(1).max(500, {
message: "sessionId must not exceed 500 characters",
}).regex(/^[a-zA-Z0-9_.\-:]+$/, {
message: "Must only contain alphanumeric, underscore, hyphen, dot, or colon characters",
});
const ToolPairSchema = z.object({
tool_name: z.string(),
tool_call_id: z.string(),
params: z.unknown(),
result: z.unknown(),
error: z.string().optional(),
timestamp: z.string(),
duration_ms: z.number().optional(),
});
/** Recent message item: user/assistant text only (no tool_call/tool_result). */
const RecentMessageSchema = z.object({
role: z.enum(["user", "assistant"]),
content: z.string(),
});
export const IngestRequestSchema = z
.object({
session_id: safeSessionId,
tool_pairs: z.array(ToolPairSchema).default([]),
/** Current user prompt that triggers L1.5 task judgment. Must be non-empty (whitespace-only is rejected). */
prompt: z.string().trim().min(1, { message: "prompt must not be empty or whitespace-only" }).optional(),
/** Recent history messages (user/assistant only, no tool calls). */
recent_messages: z.array(RecentMessageSchema).optional(),
})
.refine(
(data) => data.tool_pairs.length > 0 || (data.prompt && data.prompt.length > 0),
{ message: "Either tool_pairs must be non-empty or prompt must be provided" },
);
export type IngestRequest = z.infer<typeof IngestRequestSchema>;
/**
* Each message must have non-empty `role` and `content` fields.
* Intentionally lenient: any non-empty string role is accepted to support
* OpenAI / Anthropic / OpenClaw-wrapped formats without over-specifying.
*/
const CompactionMessageSchema = z
.record(z.string(), z.unknown())
.refine(
(msg) => typeof msg["role"] === "string" && (msg["role"] as string).length > 0,
{ message: "Each message must have a non-empty 'role' field" },
);
export const CompactionRequestSchema = z.object({
session_id: safeSessionId,
messages: z.array(CompactionMessageSchema),
ratio: z.number().min(0).max(2),
});
export type CompactionRequest = z.infer<typeof CompactionRequestSchema>;
/** Extended compaction schema with token metadata for L3 compression. */
export const CompactionRequestSchemaV2 = z.object({
session_id: safeSessionId,
messages: z.array(CompactionMessageSchema),
ratio: z.number().min(0).max(2),
context_window: z.number().int().min(1),
total_tokens: z.number().int().min(0),
message_tokens: z.array(z.number()).optional(),
});
export type CompactionRequestV2 = z.infer<typeof CompactionRequestSchemaV2>;
export const MmdQuerySchema = z.object({
session_id: safeSessionId,
limit: z.number().int().min(1).optional(),
});
+29
View File
@@ -0,0 +1,29 @@
/**
* Offload Session Utilities unified sessionId sanitization.
*
* sessionId: opaque identifier from client (free format, e.g. "main:main", "coding:session-001")
* sanitizedSessionId: filesystem-safe version used for:
* - File/directory paths (offload/{sanitizedSessionId}/)
* - Distributed lock keys
* - Task queue session identifiers
*
* Conversion: replace all non-alphanumeric/dot/hyphen/underscore chars with "_"
*/
/**
* Convert a raw sessionId to a filesystem-safe string.
* Replaces colons and other unsafe characters with underscores.
*/
export function sanitizeSessionId(sessionId: string): string {
return sessionId.replace(/[^a-zA-Z0-9._\-]/g, "_");
}
/**
* Build the offload storage base path for a session.
*/
export function buildOffloadBasePath(sessionId: string): string {
return `offload/${sanitizeSessionId(sessionId)}`;
}
// Legacy compat: keep old name as alias (will remove after all callers migrated)
export const toSessionId = sanitizeSessionId;
+132
View File
@@ -0,0 +1,132 @@
/**
* Task Transition logic manages MMD file creation/switching based on L1.5 judgment.
*/
import type { StorageAdapter } from "../core/storage/adapter.js";
import type { OffloadState, TaskJudgment, MmdMeta } from "./types.js";
/**
* Apply task transition based on L1.5 judgment.
* Mutates state in place; may create new MMD file in COS.
* @param basePath - Full storage base path, e.g. "offload/default/agent_main_tui-xxx"
*/
export async function handleTaskTransition(
state: OffloadState,
judgment: TaskJudgment,
storage: StorageAdapter,
basePath: string,
): Promise<void> {
if (judgment.taskCompleted && judgment.isLongTask && !judgment.isContinuation && judgment.newTaskLabel) {
// CASE 1: New long task → create new MMD file
const filename = await generateMmdFilename(storage, basePath, judgment.newTaskLabel);
await storage.writeFile(`${basePath}/mmds/${filename}`, "");
state.activeMmdFile = filename;
} else if (judgment.taskCompleted && judgment.isContinuation && judgment.continuationMmdFile) {
// CASE 2: Continue historical task → switch to that MMD
state.activeMmdFile = judgment.continuationMmdFile;
} else if (judgment.taskCompleted && !judgment.isLongTask) {
// CASE 3: Short task / casual chat → clear active MMD
state.activeMmdFile = null;
} else if (!judgment.taskCompleted && judgment.isLongTask && !state.activeMmdFile) {
// CASE 5: Task in progress but no active MMD yet → create one
const label = judgment.newTaskLabel || "current-task";
const filename = await generateMmdFilename(storage, basePath, label);
await storage.writeFile(`${basePath}/mmds/${filename}`, "");
state.activeMmdFile = filename;
}
// CASE 4: !taskCompleted + activeMmdFile already set → keep unchanged
}
/**
* Generate a new MMD filename with auto-incrementing sequence number.
* Format: "003-refactor-api.mmd"
*/
async function generateMmdFilename(
storage: StorageAdapter,
basePath: string,
label: string,
): Promise<string> {
const mmdsPrefix = `${basePath}/mmds/`;
const existingFiles = await storage.readdirNames(mmdsPrefix, ".mmd");
let maxSeq = 0;
for (const f of existingFiles) {
const match = f.match(/^(\d+)-/);
if (match) {
const seq = parseInt(match[1], 10);
if (seq > maxSeq) maxSeq = seq;
}
}
const nextSeq = String(maxSeq + 1).padStart(3, "0");
const safeLabel = label
.toLowerCase()
.replace(/[^a-z0-9-]/g, "-")
.replace(/-+/g, "-")
.replace(/^-|-$/g, "")
.slice(0, 30);
return `${nextSeq}-${safeLabel || "task"}.mmd`;
}
/**
* Extract MmdMeta from MMD file content (parses %%{ ... }%% header).
*/
export function extractMmdMeta(filename: string, content: string): MmdMeta {
const defaults: MmdMeta = {
filename,
taskGoal: "",
doneCount: 0,
doingCount: 0,
todoCount: 0,
updatedTime: null,
nodeSummaries: [],
};
if (!content) return defaults;
// Parse %%{ ... }%% metadata line
const metaMatch = content.match(/%%\{([\s\S]*?)\}%%/);
if (metaMatch) {
try {
const meta = JSON.parse(`{${metaMatch[1]}}`);
defaults.taskGoal = meta.taskGoal ?? "";
defaults.updatedTime = meta.updatedTime ?? null;
} catch {
// ignore parse errors
}
}
// Count node statuses and extract summaries
const nodeRegex = /(\w[\w-]*)\["([^"]*?)"\]/g;
let nodeMatch;
while ((nodeMatch = nodeRegex.exec(content)) !== null) {
const nodeId = nodeMatch[1];
const nodeContent = nodeMatch[2];
const statusM = nodeContent.match(/status:\s*(done|doing|todo|paused|blocked)/i);
const summaryM = nodeContent.match(/summary:\s*(.+?)(?:<br\/>|$)/i);
if (statusM) {
const s = statusM[1].toLowerCase();
if (s === "done") defaults.doneCount++;
else if (s === "doing") defaults.doingCount++;
else if (s === "todo") defaults.todoCount++;
defaults.nodeSummaries!.push({
nodeId,
status: s,
summary: summaryM?.[1]?.trim() ?? "",
});
}
}
// Fallback: if regex didn't catch statuses, try simple pattern
if (defaults.doneCount === 0 && defaults.doingCount === 0 && defaults.todoCount === 0) {
const statusMatches = content.matchAll(/status:\s*(done|doing|todo|paused|blocked)/gi);
for (const m of statusMatches) {
const s = m[1].toLowerCase();
if (s === "done") defaults.doneCount++;
else if (s === "doing") defaults.doingCount++;
else if (s === "todo") defaults.todoCount++;
}
}
return defaults;
}
+163
View File
@@ -0,0 +1,163 @@
/**
* Offload Server Independent type definitions.
* This module does NOT import from src/offload/.
*/
// ─── ToolPair (ingest 写入 pending.jsonl) ────────────────────────────────────
export interface ToolPair {
toolName: string;
toolCallId: string;
params: unknown;
result: unknown;
error?: string;
timestamp: string;
durationMs?: number;
}
// ─── OffloadEntry (L1 产出, entries.jsonl 每行) ──────────────────────────────
export interface OffloadEntry {
tool_call_id: string;
tool_call: string;
summary: string;
timestamp: string;
score: number;
node_id: string | null;
/** Full relative path to ref file storing original tool result (e.g. "offload/{sessionId}/refs/call_214.md"), readable via tdai_read_cos */
result_ref?: string;
}
// ─── TaskJudgment (L1.5 产出) ────────────────────────────────────────────────
export interface TaskJudgment {
taskCompleted: boolean;
isLongTask: boolean;
isContinuation: boolean;
continuationMmdFile?: string;
newTaskLabel?: string;
}
// ─── TaskBoundary (L1.5 写入 state.boundaries) ──────────────────────────────
export interface TaskBoundary {
targetMmd: string | null;
timestamp: string;
}
// ─── MmdMeta (从 MMD 文件首行 %%{...}%% 解析) ──────────────────────────────
export interface MmdMeta {
filename: string;
taskGoal: string;
doneCount: number;
doingCount: number;
todoCount: number;
updatedTime?: string | null;
nodeSummaries?: Array<{ nodeId: string; status: string; summary: string }>;
}
// ─── OffloadState (state.json) ───────────────────────────────────────────────
export interface OffloadState {
activeMmdFile: string | null;
boundaries: TaskBoundary[];
lastL15CreatedAt: number;
}
// ─── CompactState (compact-state.json) ───────────────────────────────────────
export interface CompactState {
confirmedOffloadIds: string[];
deletedOffloadIds: string[];
lastCompactedAt: string;
}
// ─── L2 Parsed Response ──────────────────────────────────────────────────────
export interface L2ParsedResponse {
fileAction: "write" | "replace";
mmdContent?: string;
replaceBlocks?: Array<{
startLine: number;
endLine: number;
content: string;
}>;
nodeMapping: Record<string, string>;
}
// ─── Configuration ───────────────────────────────────────────────────────────
export interface OffloadExecutorConfig {
forceTriggerThreshold: number;
pendingMaxAgeSeconds: number;
l1Model: string;
l1Temperature: number;
l1MaxTokens: number;
l1TimeoutMs: number;
l15Model: string;
l15Temperature: number;
l15MaxTokens: number;
l15TimeoutMs: number;
l2Model: string;
l2Temperature: number;
l2MaxTokens: number;
l2TimeoutMs: number;
l2NullThreshold: number;
mildOffloadRatio: number;
aggressiveCompressRatio: number;
emergencyCompressRatio: number;
maxRetries: number;
}
export function defaultOffloadConfig(): OffloadExecutorConfig {
return {
forceTriggerThreshold: 4,
pendingMaxAgeSeconds: 30,
l1Model: "", // uses gateway llm.model
l1Temperature: 0.3,
l1MaxTokens: 8000,
l1TimeoutMs: 120_000,
l15Model: "", // uses gateway llm.model
l15Temperature: 0.2,
l15MaxTokens: 3000,
l15TimeoutMs: 120_000,
l2Model: "", // uses gateway llm.model
l2Temperature: 0.4,
l2MaxTokens: 16000,
l2TimeoutMs: 120_000,
l2NullThreshold: 6,
mildOffloadRatio: 0.5,
aggressiveCompressRatio: 0.85,
emergencyCompressRatio: 0.95,
maxRetries: 3,
};
}
// ─── Default empty state ─────────────────────────────────────────────────────
export function defaultOffloadState(): OffloadState {
return {
activeMmdFile: null,
boundaries: [],
lastL15CreatedAt: 0,
};
}
export function defaultCompactState(): CompactState {
return {
confirmedOffloadIds: [],
deletedOffloadIds: [],
lastCompactedAt: "",
};
}
+94 -3
View File
@@ -42,6 +42,9 @@ export interface TaskExecutor {
executeL2(task: TaskPayload, signal?: AbortSignal): Promise<void>;
executeL3(task: TaskPayload, signal?: AbortSignal): Promise<void>;
executeFlush?(task: TaskPayload, signal?: AbortSignal): Promise<void>;
executeOffloadL1?(task: TaskPayload, signal?: AbortSignal): Promise<void>;
executeOffloadL15?(task: TaskPayload, signal?: AbortSignal): Promise<void>;
executeOffloadL2?(task: TaskPayload, signal?: AbortSignal): Promise<void>;
}
export interface PipelineWorkerConfig {
@@ -249,11 +252,67 @@ export class PipelineWorker {
const lockKey = this.getLockKey(task);
const retryCount = (task.data?.retryCount as number) ?? 0;
// Lock-free path: offload-l1 doesn't need distributed lock
if (lockKey === null) {
this.runningTasks.set(task.id, task);
try {
await this.executeTask(task, undefined);
// ACK
const msgId = (task as any)._msgId;
if (msgId) await this.backend.ackTask(msgId);
this.metrics.tasksCompleted++;
this.logger?.debug?.(`${TAG} Task completed (lock-free): ${task.type} [${task.instanceId}/${task.sessionId}]`);
} catch (err) {
const errMsg = err instanceof Error ? err.message : String(err);
this.metrics.tasksFailed++;
if (retryCount < this.config.maxRetries) {
const delay = this.config.retryBaseDelayMs * Math.pow(3, retryCount);
this.logger.warn(`${TAG} Task failed (lock-free, retry ${retryCount + 1}/${this.config.maxRetries}, delay=${delay}ms): ${errMsg}`);
const msgId = (task as any)._msgId;
if (msgId) { try { await this.backend.ackTask(msgId); } catch { /* best effort */ } }
await this.sleep(delay);
await this.reEnqueue(task, retryCount + 1);
this.metrics.tasksRetried++;
} else {
await this.moveToDeadLetter(task, errMsg, retryCount);
}
} finally {
this.runningTasks.delete(task.id);
// Deferred enqueue (same as locked path)
const deferred = (task as any)._deferredEnqueue as TaskPayload[] | undefined;
if (deferred?.length) {
for (const dTask of deferred) {
try {
await this.backend.enqueueTask(dTask);
this.logger?.debug?.(`${TAG} Deferred enqueue: ${dTask.type} [${dTask.id}]`);
} catch (err) {
this.logger?.warn?.(`${TAG} Deferred enqueue failed: ${err instanceof Error ? err.message : String(err)}`);
}
}
}
}
return;
}
// Step 1: 抢分布式锁
const locked = await this.backend.acquireLock(lockKey, this.config.workerId, this.config.lockTtlMs);
if (!locked) {
this.metrics.lockConflicts++;
// offload-l2: skip immediately on lock conflict (idempotent timer will re-trigger)
if (task.type === "offload-l2") {
this.logger?.debug?.(`${TAG} Lock conflict [offload-l2] (task=${task.id}): ${lockKey}, skip (timer will re-trigger)`);
const msgId = (task as any)._msgId;
if (msgId) {
try { await this.backend.ackTask(msgId); } catch { /* best effort */ }
}
return;
}
// Lock conflict: current coroutine waits locally (no re-enqueue to stream).
// Exponential backoff: 200ms → 600ms → 1.8s → 5s (capped), retry until lockTtlMs exhausted.
// 旧版本固定 sleep(5000) 在 instance 级锁下会造成排队体感差(同 instance 多 session 累积秒级延迟);
@@ -265,14 +324,14 @@ export class PipelineWorker {
let delay = 200;
while (Date.now() < deadline && this.running) {
attempt++;
this.logger?.debug?.(`${TAG} Lock conflict: ${lockKey}, retry ${attempt} after ${delay}ms`);
this.logger?.debug?.(`${TAG} Lock conflict [${task.type}] (task=${task.id}): ${lockKey}, retry ${attempt} after ${delay}ms`);
await this.sleep(delay);
acquired = await this.backend.acquireLock(lockKey, this.config.workerId, this.config.lockTtlMs);
if (acquired) break;
delay = Math.min(delay * 3, 5000);
}
if (!acquired) {
this.logger?.warn?.(`${TAG} Lock conflict timeout: ${lockKey}, dropping task`);
this.logger?.warn?.(`${TAG} Lock conflict timeout [${task.type}] (task=${task.id}): ${lockKey}, dropping task`);
// CR-1 fix: ACK to prevent stale recovery from re-claiming this message in an
// infinite loop. Without it, XPENDING keeps returning this msgId every
// pendingRecoveryIntervalMs, exhausting worker slots.
@@ -396,6 +455,20 @@ export class PipelineWorker {
this.activeLocks.delete(lockKey);
this.runningTasks.delete(task.id);
try { await this.backend.releaseLock(lockKey, this.config.workerId); } catch { /* best effort */ }
// Step 7: 延迟入队 — executor 可通过 task._deferredEnqueue 暂存需要在锁释放后才入队的任务,
// 避免新任务立即被消费时因同 session 锁仍被持有而产生不必要的锁冲突。
const deferred = (task as any)._deferredEnqueue as TaskPayload[] | undefined;
if (deferred?.length) {
for (const dTask of deferred) {
try {
await this.backend.enqueueTask(dTask);
this.logger?.debug?.(`${TAG} Deferred enqueue: ${dTask.type} [${dTask.id}]`);
} catch (err) {
this.logger?.warn?.(`${TAG} Deferred enqueue failed: ${err instanceof Error ? err.message : String(err)}`);
}
}
}
}
}
@@ -405,6 +478,9 @@ export class PipelineWorker {
case "L2": return this.executor.executeL2(task, signal);
case "L3": return this.executor.executeL3(task, signal);
case "flush": return this.executor.executeFlush?.(task, signal) ?? this.executor.executeL1(task, signal);
case "offload-l1": return this.executor.executeOffloadL1?.(task, signal);
case "offload-l15": return this.executor.executeOffloadL15?.(task, signal);
case "offload-l2": return this.executor.executeOffloadL2?.(task, signal);
default:
this.logger.warn(`${TAG} Unknown task type: ${task.type}`);
}
@@ -494,7 +570,22 @@ export class PipelineWorker {
* pod lock key pod
* 缓解: 升级时停所有 worker pending
*/
private getLockKey(task: TaskPayload): string {
private getLockKey(task: TaskPayload): string | null {
// offload-l1 is lock-free: rename guarantees exclusive file ownership,
// appendFile is atomic (O_APPEND), and state.json is read-only for L1.
if (task.type === "offload-l1") return null;
// offload-l2: per-MMD lock so different MMDs can be processed concurrently.
if (task.type === "offload-l2") {
const mmdFile = (task.data as any)?.targetMmdFile ?? "default";
return `pipeline:{${task.instanceId}}:offload-l2:${mmdFile}`;
}
// offload-l15: lock-free at worker level. The executor acquires a short
// lock only during the final write phase (state.json update), allowing
// multiple L1.5 LLM calls to run concurrently without blocking each other.
if (task.type === "offload-l15") return null;
if (this.config.lockGranularity === "instance") {
return `pipeline:{${task.instanceId}}`;
}
+41 -9
View File
@@ -152,7 +152,7 @@ export class TimerScanner {
}
for (const entry of expired) {
const { instanceId, sessionId, taskType, priority } = this.parseShardMember(entry.member);
const { instanceId, sessionId, taskType, priority, timerType } = this.parseShardMember(entry.member);
const task: TaskPayload = {
id: `${taskType}-${instanceId.slice(-8)}-${sessionId.slice(-8)}-${now}`,
@@ -161,7 +161,7 @@ export class TimerScanner {
sessionId,
priority,
createdAt: now,
data: { triggeredBy: "timer_scanner", timerMember: `${sessionId}:${taskType === "L1" ? "L1_idle" : taskType === "L2" ? "L2_schedule" : "L3"}`, instanceId },
data: { triggeredBy: "timer_scanner", timerMember: `${sessionId}:${timerType}`, instanceId },
};
await this.backend.enqueueTask(task);
@@ -191,7 +191,7 @@ export class TimerScanner {
* Parse shard member format: "{instanceId}\x00{sessionId}:{timerType}"
* Example: "mem-j4wjesud\x00sess_001:L1_idle" { instanceId: "mem-j4wjesud", sessionId: "sess_001", taskType: "L1" }
*/
private parseShardMember(member: string): { instanceId: string; sessionId: string; taskType: "L1" | "L2" | "L3" | "flush"; priority: number } {
private parseShardMember(member: string): { instanceId: string; sessionId: string; taskType: TaskPayload["type"]; priority: number; timerType: string } {
const sep = member.indexOf("\x00");
let instanceId: string;
let rest: string;
@@ -206,19 +206,51 @@ export class TimerScanner {
rest = member.slice(firstColon + 1);
}
// rest = "sessionId:timerType" (e.g. "sess_001:L1_idle", "sess_001:L2_schedule")
// rest = timer member after instanceId separator
// New unified format: "offload-{type}:{embeddedInstanceId}:{sessionId}[:{extra}]" (prefix-based)
// Legacy format: "sessionId:L1_idle" or "sessionId:L2_schedule"
// Check for offload prefix format first
if (rest.startsWith("offload-l15:")) {
// Skip embedded instanceId: "offload-l15:{instanceId}:{sessionId}"
const afterPrefix = rest.slice("offload-l15:".length);
const colonIdx = afterPrefix.indexOf(":");
const sessionId = colonIdx > 0 ? afterPrefix.slice(colonIdx + 1) : afterPrefix;
return { instanceId, sessionId, taskType: "offload-l15", priority: 0, timerType: rest };
}
if (rest.startsWith("offload-l2:")) {
// Skip embedded instanceId: "offload-l2:{instanceId}:{sessionId}[:{mmdFile}]"
const afterPrefix = rest.slice("offload-l2:".length);
const colonIdx = afterPrefix.indexOf(":");
let sessionId = colonIdx > 0 ? afterPrefix.slice(colonIdx + 1) : afterPrefix;
// Strip trailing ":{mmdFile}" from sessionId
if (sessionId.endsWith(".mmd")) {
const lastColon = sessionId.lastIndexOf(":");
if (lastColon > 0) sessionId = sessionId.slice(0, lastColon);
}
return { instanceId, sessionId, taskType: "offload-l2", priority: 1, timerType: rest };
}
if (rest.startsWith("offload-l1:")) {
// Skip embedded instanceId: "offload-l1:{instanceId}:{sessionId}"
const afterPrefix = rest.slice("offload-l1:".length);
const colonIdx = afterPrefix.indexOf(":");
const sessionId = colonIdx > 0 ? afterPrefix.slice(colonIdx + 1) : afterPrefix;
return { instanceId, sessionId, taskType: "offload-l1", priority: 0, timerType: rest };
}
// For non-offload types, use lastColon (original logic)
const lastColon = rest.lastIndexOf(":");
if (lastColon <= 0) {
return { instanceId, sessionId: rest, taskType: "L1", priority: 0 };
return { instanceId, sessionId: rest, taskType: "L1", priority: 0, timerType: "L1_idle" };
}
const sessionId = rest.slice(0, lastColon);
const timerType = rest.slice(lastColon + 1);
if (timerType.startsWith("L1")) return { instanceId, sessionId, taskType: "L1", priority: 0 };
if (timerType.startsWith("L2")) return { instanceId, sessionId, taskType: "L2", priority: 1 };
if (timerType.startsWith("L3")) return { instanceId, sessionId, taskType: "L3", priority: 2 };
return { instanceId, sessionId, taskType: "flush", priority: 0 };
if (timerType.startsWith("L1")) return { instanceId, sessionId, taskType: "L1", priority: 0, timerType };
if (timerType.startsWith("L2")) return { instanceId, sessionId, taskType: "L2", priority: 1, timerType };
if (timerType.startsWith("L3")) return { instanceId, sessionId, taskType: "L3", priority: 2, timerType };
return { instanceId, sessionId, taskType: "flush", priority: 0, timerType };
}
}
+15 -16
View File
@@ -5,7 +5,7 @@
* Guarantees:
* 1. Blank conversation history (temporary session file)
* 2. Independent system prompt (only the task prompt)
* 3. No tool calls (tools restricted to minimal read-only set to avoid empty tools[] rejection by some providers)
* 3. No tool calls when enableTools=false (disableTools:true no tool definitions sent to API)
* 4. No contamination from the main agent's context
*/
@@ -17,6 +17,7 @@ import { fileURLToPath, pathToFileURL } from "node:url";
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/core";
import { getEnv } from "./env.js";
import { report } from "../core/report/reporter.js";
import type { Logger } from "../core/types.js";
/**
* Resolve a preferred temporary directory for memory-tdai operations.
@@ -49,12 +50,7 @@ function resolveOpenClawTmpDir(): string {
const TAG = "[memory-tdai] [runner]";
interface RunnerLogger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
type RunnerLogger = Logger;
// Dynamic import type — runEmbeddedPiAgent is an internal API
// Prefer the public plugin runtime signature so host-injected runtimes stay assignable.
@@ -394,10 +390,11 @@ export class CleanContextRunner {
},
tools: {
...((this.options.config as Record<string, unknown>)?.tools as Record<string, unknown> | undefined),
// When enableTools=false we still keep one lightweight read-only tool
// so that the tools array sent to the API is non-empty.
// Some providers (e.g. qwencode) reject tools:[] with minItems:1 validation.
allow: this.options.enableTools ? ["read", "write", "edit"] : ["read"],
// When enableTools=true, restrict to the minimal set needed for
// scene extraction (read/write/edit).
// When enableTools=false, pass an empty allow list — disableTools:true
// will prevent tools from being sent to the API entirely.
allow: this.options.enableTools ? ["read", "write", "edit"] : [],
},
// Override the full agent system prompt with the caller's extraction-specific
// system prompt. This replaces OpenClaw's default system prompt (identity,
@@ -455,11 +452,13 @@ export class CleanContextRunner {
runId,
provider: this.resolvedProvider,
model: this.resolvedModel,
// Do NOT pass disableTools:true that produces tools:[] which some
// providers (qwencode) reject with "[] is too short - 'tools'".
// Instead rely on cleanConfig.tools.allow to restrict the tool set
// to a minimal read-only tool (when enableTools=false).
disableTools: false,
// When enableTools=false, pass disableTools:true so that no tool
// definitions are sent to the API. This avoids polluting the LLM
// context with tool schemas and prevents the model from attempting
// tool calls during pure text extraction tasks.
// If a provider (e.g. qwencode) rejects empty tools[], users should
// switch to StandaloneLLMRunner via LLM configuration instead.
disableTools: !this.options.enableTools,
extraSystemPrompt: effectiveSystemPrompt,
streamParams: {
maxTokens: params.maxTokens,
+1 -7
View File
@@ -3,13 +3,7 @@ import path from "node:path";
import type { IMemoryStore } from "../core/store/types.js";
import { ManagedTimer } from "./managed-timer.js";
interface Logger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
import type { Logger } from "../core/types.js";
export interface MemoryCleanerOptions {
baseDir: string;
+3 -6
View File
@@ -37,6 +37,7 @@ import { PersonaTrigger } from "../core/persona/persona-trigger.js";
import { PersonaGenerator } from "../core/persona/persona-generator.js";
import { pullProfilesToLocal, syncLocalProfilesToStore } from "../core/profile/profile-sync.js";
import type { StorageAdapter } from "../core/storage/adapter.js";
import type { Logger } from "../core/types.js";
const TAG = "[memory-tdai] [pipeline-factory]";
@@ -72,12 +73,8 @@ function supportsProfileSyncWrite(store?: IMemoryStore): boolean {
// Logger interface
// ============================
export interface PipelineLogger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
/** @deprecated Use `Logger` from `../core/types.js` directly. */
export type PipelineLogger = Logger;
// ============================
// Factory options
+1 -7
View File
@@ -81,18 +81,12 @@ import { SessionFilter } from "./session-filter.js";
import { ManagedTimer } from "./managed-timer.js";
import { SerialQueue } from "./serial-queue.js";
import { report } from "../core/report/reporter.js";
import type { Logger } from "../core/types.js";
// ============================
// Types
// ============================
interface Logger {
debug?: (message: string) => void;
info: (message: string) => void;
warn: (message: string) => void;
error: (message: string) => void;
}
/** A single captured message ready for L1 processing. */
export interface CapturedMessage {
role: "user" | "assistant" | "tool";