feat(embedding): support ZeroEntropy native embed API (#137)

* feat(embedding): support ZeroEntropy native embed API

ZeroEntropy's embedding service is reachable but not OpenAI-shaped:
its endpoint is POST /v1/models/embed (not /v1/embeddings), the
request body requires an `input_type` ("query" or "document") and
rejects an explicit `dimensions` field, and the response is wrapped
as `{results: [{embedding}]}` instead of OpenAI's
`{data: [{index, embedding}]}`.

Branch on `provider === "zeroentropy"` inside OpenAIEmbeddingService
rather than introducing a new service class:

- `/models/embed` is used instead of `/embeddings` (also honoured
  by the qclaw proxy path so users can still front it).
- request body sets `input_type: "query"` and omits `dimensions`
  (zembed-1 is fixed-dimension and rejects explicit overrides).
- response parsing decodes the `results[]` envelope and preserves
  input order via array index rather than OpenAI's `index` field.

`openclaw.plugin.json` updates the `embedding.provider` description to
list `zeroentropy` as a recognized value.

Everything else (timeout, retries, batching, char-cap truncation,
sanitize+normalize) is shared with the OpenAI path — only the wire
format changes.

Fixes #68

Signed-off-by: 李冠辰 <liguanchen@xiaomi.com>

* refactor(embedding): split ZeroEntropy into its own EmbeddingService

Addresses @Xuruida's review feedback on PR #137: instead of branching
OpenAIEmbeddingService._callApi on `providerName === "zeroentropy"`,
extract the shared HTTP layer and give ZeroEntropy its own class.

This keeps OpenAIEmbeddingService focused on the OpenAI-compatible
wire format and stops provider-specific protocol differences from
accumulating inside a single class.

Changes:

- New module-level helpers (provider-agnostic):
  * `truncateEmbeddingInputs(texts, maxInputChars, logger)` — was
    previously a private method duplicated logic-side.
  * `postEmbeddingRequest({fetchUrl, headers, body, timeoutMs})` —
    owns fetch + AbortController timeout + exponential-backoff
    retries + the EmbeddingApiError non-retry rule for 4xx (except
    429). Returns the parsed JSON; callers own response parsing.

- `OpenAIEmbeddingService._callApi` is back to OpenAI-only: builds
  `{input, model, dimensions?}`, posts to `/embeddings`, parses
  `{data: [{index, embedding}]}` with index-sort. The qclaw proxy
  branch is unchanged. No more `isZeroEntropy` reads.

- New `ZeroEntropyEmbeddingService implements EmbeddingService`:
  - Same `OpenAIEmbeddingConfig` shape (baseUrl / apiKey / model /
    dimensions / maxInputChars / timeoutMs are identical on the
    wire), but encodes ZeroEntropy's wire format independently:
    posts to `${baseUrl}/models/embed`, sends `input_type: "query"`,
    omits `dimensions` (zembed-1 is fixed-dim), parses
    `{results: [{embedding}]}` preserving array order.
  - Doesn't carry the OpenAI-only knobs (`sendDimensions`,
    `proxyUrl`, qclaw) so the class surface stays small.

- `createEmbeddingService` factory: new `provider === "zeroentropy"`
  branch placed before the OpenAI-compatible catch-all so it wins
  the dispatch.

Behaviour preserved:
- All existing OpenAI / DeepSeek / Azure / qclaw deployments hit
  exactly the same wire calls as before this PR was opened.
- ZeroEntropy wire format (URL + body + response) matches the
  original PR's branched implementation byte for byte.
- Timeout, retry, error mapping, batch splitting, char-cap
  truncation, sanitize+normalize are shared via the module helpers
  so the two services can't diverge by accident.

Signed-off-by: 李冠辰 <liguanchen@xiaomi.com>

* fix(embedding): forward dimensions to ZeroEntropy when sendDimensions

Addresses @Xuruida's follow-up review on PR #137. The previous commit
silently dropped `dimensions` from every ZeroEntropy request based on
the wrong claim that zembed-1 is fixed-dim.

Verified against
https://docs.zeroentropy.dev/api-reference/models/embed: `dimensions`
is an optional integer; for zembed-1 the accepted set is
[2560, 1280, 640, 320, 160, 80, 40] (Matryoshka truncation). Not
forwarding it left users no way to pick a smaller dim end-to-end —
the wire payload always used the model default while the Float32Array
return type and getDimensions() still reported config.dimensions, so
the runtime vector size and the configured one could disagree.

- ZeroEntropyEmbeddingService now mirrors OpenAIEmbeddingService:
  reads `config.sendDimensions ?? true` into a `sendDimensions` field
  and emits `dimensions: this.dims` in the request body when set.
- Class docstring corrected — the "rejects explicit overrides" line
  is replaced with the documented Matryoshka set. Server still
  rejects out-of-set values, but that's a config choice the user
  owns; we forward verbatim instead of clamping silently.

Signed-off-by: 李冠辰 <liguanchen@xiaomi.com>

---------

Signed-off-by: 李冠辰 <liguanchen@xiaomi.com>

openclaw.plugin.json

@@ -87,7 +87,7 @@
         "description": "向量搜索 (Embedding) 配置",
         "properties": {
           "enabled": { "type": "boolean", "default": true, "description": "是否启用向量搜索（若 provider=none，则实际会被禁用）" },
-          "provider": { "type": "string", "default": "none", "description": "Embedding 服务提供者：填写兼容 OpenAI API 的远端服务名称（如 openai、deepseek 等）；不填或填 none 则禁用向量搜索" },
+          "provider": { "type": "string", "default": "none", "description": "Embedding 服务提供者：填写兼容 OpenAI API 的远端服务名称（如 openai、deepseek 等）；填 zeroentropy 走 ZeroEntropy 原生 /v1/models/embed 协议（zembed-1 等模型）；不填或填 none 则禁用向量搜索" },
           "proxyUrl": { "type": "string", "description": "本地代理地址（仅 provider=qclaw 时必填）。配置后 embedding 请求将通过该代理转发，原始 baseUrl 作为 Remote-URL 头传递" },
           "baseUrl": { "type": "string", "description": "API Base URL（必填）：填写对应 provider 的 API 地址" },
           "apiKey": { "type": "string", "description": "API Key（必填）" },

src/core/store/embedding.ts

@@ -399,6 +399,101 @@ interface OpenAIEmbeddingResponse {
   };
 }
 
+/**
+ * ZeroEntropy's `/v1/models/embed` returns input order via `results[i]`
+ * (no `index` field) and omits the OpenAI `data` envelope. See:
+ *   https://docs.zeroentropy.dev/api-reference/models/embed
+ */
+interface ZeroEntropyEmbeddingResponse {
+  results: Array<{
+    embedding: number[];
+  }>;
+}
+
+// ============================
+// Shared HTTP helpers (provider-agnostic)
+// ============================
+
+/**
+ * Truncate every text to `maxInputChars` (when set), emitting one warning
+ * per text that exceeded the limit. Returns the input array untouched when
+ * no limit is configured.
+ */
+function truncateEmbeddingInputs(
+  texts: string[],
+  maxInputChars: number | undefined,
+  logger?: Logger,
+): string[] {
+  if (!maxInputChars) return texts;
+  return texts.map((text) => {
+    if (text.length <= maxInputChars) return text;
+    logger?.warn?.(
+      `${TAG} Input truncated from ${text.length} to ${maxInputChars} chars (maxInputChars limit)`,
+    );
+    return text.slice(0, maxInputChars);
+  });
+}
+
+/**
+ * POST a remote embedding request with the project's standard timeout +
+ * retry behaviour, returning the parsed JSON body. Provider-specific
+ * services own body construction and response shape — this helper handles
+ * fetch, abort-on-timeout, exponential backoff, and the `EmbeddingApiError`
+ * non-retry rule for 4xx responses (except 429).
+ */
+async function postEmbeddingRequest(params: {
+  fetchUrl: string;
+  headers: Record<string, string>;
+  body: Record<string, unknown>;
+  timeoutMs: number;
+}): Promise<unknown> {
+  const { fetchUrl, headers, body, timeoutMs } = params;
+  let lastError: Error | undefined;
+  for (let attempt = 0; attempt <= MAX_RETRIES; attempt++) {
+    try {
+      const controller = new AbortController();
+      const timeoutId = setTimeout(() => controller.abort(), timeoutMs);
+      try {
+        const resp = await fetch(fetchUrl, {
+          method: "POST",
+          headers,
+          body: JSON.stringify(body),
+          signal: controller.signal,
+        });
+        if (!resp.ok) {
+          const errBody = await resp.text().catch(() => "(unable to read body)");
+          const err = new EmbeddingApiError(
+            `Embedding API error: HTTP ${resp.status} ${resp.statusText} — ${errBody.slice(0, 500)}`,
+            resp.status,
+          );
+          // Don't retry 4xx client errors (except 429 rate limit).
+          if (resp.status >= 400 && resp.status < 500 && resp.status !== 429) {
+            throw err;
+          }
+          lastError = err;
+          continue;
+        }
+        return await resp.json();
+      } finally {
+        clearTimeout(timeoutId);
+      }
+    } catch (err) {
+      // Non-retryable errors (4xx client errors) — rethrow immediately
+      if (err instanceof EmbeddingApiError && err.isClientError()) {
+        throw err;
+      }
+      lastError = err instanceof Error ? err : new Error(String(err));
+      // AbortError = timeout, retry
+      if (attempt < MAX_RETRIES) {
+        // Exponential backoff: 500ms, 1000ms
+        const delay = 500 * (attempt + 1);
+        await new Promise((r) => setTimeout(r, delay));
+      }
+    }
+  }
+  throw lastError ?? new Error("Embedding API call failed after retries");
+}
+
 export class OpenAIEmbeddingService implements EmbeddingService {
   private readonly baseUrl: string;
   private readonly apiKey: string;
@@ -502,7 +597,7 @@ export class OpenAIEmbeddingService implements EmbeddingService {
       body.dimensions = this.dims;
     }
 
-    // Determine fetch URL and headers based on proxy mode
+    // Determine fetch URL and headers based on proxy mode.
     const useProxy = this.providerName === "qclaw" && !!this.proxyUrl;
     const fetchUrl = useProxy ? this.proxyUrl! : `${this.baseUrl}/embeddings`;
     const headers: Record<string, string> = {
@@ -516,63 +611,161 @@ export class OpenAIEmbeddingService implements EmbeddingService {
       );
     }
 
-    // Retry loop with timeout
+    const json = (await postEmbeddingRequest({
-    let lastError: Error | undefined;
+      fetchUrl,
-    for (let attempt = 0; attempt <= MAX_RETRIES; attempt++) {
+      headers,
-      try {
+      body,
-        const controller = new AbortController();
+      timeoutMs: timeoutOverride ?? this.timeoutMs,
-        const timeoutId = setTimeout(() => controller.abort(), timeoutOverride ?? this.timeoutMs);
+    })) as OpenAIEmbeddingResponse;
 
-        try {
+    if (!json.data || !Array.isArray(json.data)) {
-          const resp = await fetch(fetchUrl, {
+      throw new Error("Embedding API returned unexpected format: missing 'data' array");
-            method: "POST",
-            headers,
-            body: JSON.stringify(body),
-            signal: controller.signal,
-          });
-
-          if (!resp.ok) {
-            const errBody = await resp.text().catch(() => "(unable to read body)");
-            const err = new EmbeddingApiError(
-              `Embedding API error: HTTP ${resp.status} ${resp.statusText} — ${errBody.slice(0, 500)}`,
-              resp.status,
-            );
-            // Don't retry on 4xx client errors (except 429 rate limit)
-            if (resp.status >= 400 && resp.status < 500 && resp.status !== 429) {
-              throw err;
-            }
-            lastError = err;
-            continue;
-          }
-
-          const json = (await resp.json()) as OpenAIEmbeddingResponse;
-
-          if (!json.data || !Array.isArray(json.data)) {
-            throw new Error("Embedding API returned unexpected format: missing 'data' array");
-          }
-
-          // Sort by index to ensure correct order, then sanitize+normalize for consistency with local provider
-          const sorted = [...json.data].sort((a, b) => a.index - b.index);
-          return sorted.map((d) => sanitizeAndNormalize(d.embedding));
-        } finally {
-          clearTimeout(timeoutId);
-        }
-      } catch (err) {
-        // Non-retryable errors (4xx client errors) — rethrow immediately
-        if (err instanceof EmbeddingApiError && err.isClientError()) {
-          throw err;
-        }
-        lastError = err instanceof Error ? err : new Error(String(err));
-        // AbortError = timeout, retry
-        if (attempt < MAX_RETRIES) {
-          // Exponential backoff: 500ms, 1000ms
-          const delay = 500 * (attempt + 1);
-          await new Promise((r) => setTimeout(r, delay));
-        }
-      }
     }
 
-    throw lastError ?? new Error("Embedding API call failed after retries");
+    // Sort by index to ensure correct order, then sanitize+normalize for consistency with local provider.
+    const sorted = [...json.data].sort((a, b) => a.index - b.index);
+    return sorted.map((d) => sanitizeAndNormalize(d.embedding));
+  }
+}
+
+// ============================
+// ZeroEntropy embedding service
+// ============================
+
+/**
+ * ZeroEntropy native embedding adapter.
+ *
+ * Reuses {@link OpenAIEmbeddingConfig} for the wire-config shape (baseUrl /
+ * apiKey / model / dimensions / sendDimensions are identical), but the wire
+ * format diverges in three places, so we keep this provider on its own class
+ * instead of branching {@link OpenAIEmbeddingService}:
+ *
+ * 1. Endpoint is `${baseUrl}/models/embed` (not `/embeddings`).
+ * 2. Request body requires `input_type` (`"query"` or `"document"`).
+ *    `dimensions` is optional — for `zembed-1` the accepted values are the
+ *    Matryoshka set [2560, 1280, 640, 320, 160, 80, 40]; any other value is
+ *    rejected by the server. The config's `sendDimensions` flag (default
+ *    true) controls whether it is forwarded, matching the OpenAI path.
+ * 3. Response envelope is `{ results: [{ embedding }] }` and preserves
+ *    input order via array position rather than an `index` field.
+ *
+ * Everything else (timeout, retry, batching, char-cap truncation,
+ * sanitize+normalize) is shared via the module-level
+ * `postEmbeddingRequest` / `truncateEmbeddingInputs` helpers. See
+ * https://docs.zeroentropy.dev/api-reference/models/embed and issue #68.
+ */
+export class ZeroEntropyEmbeddingService implements EmbeddingService {
+  private readonly baseUrl: string;
+  private readonly apiKey: string;
+  private readonly model: string;
+  private readonly dims: number;
+  private readonly sendDimensions: boolean;
+  private readonly maxInputChars?: number;
+  private readonly timeoutMs: number;
+  private readonly logger?: Logger;
+
+  constructor(config: OpenAIEmbeddingConfig, logger?: Logger) {
+    if (!config.apiKey) {
+      throw new Error("ZeroEntropyEmbeddingService: apiKey is required");
+    }
+    if (!config.baseUrl) {
+      throw new Error("ZeroEntropyEmbeddingService: baseUrl is required");
+    }
+    if (!config.model) {
+      throw new Error("ZeroEntropyEmbeddingService: model is required");
+    }
+    if (!config.dimensions || config.dimensions <= 0) {
+      throw new Error("ZeroEntropyEmbeddingService: dimensions is required (must be a positive integer)");
+    }
+    this.baseUrl = config.baseUrl.replace(/\/+$/, "");
+    this.apiKey = config.apiKey;
+    this.model = config.model;
+    this.dims = config.dimensions;
+    this.sendDimensions = config.sendDimensions ?? true;
+    this.maxInputChars = config.maxInputChars && config.maxInputChars > 0 ? config.maxInputChars : undefined;
+    this.timeoutMs = config.timeoutMs && config.timeoutMs > 0 ? config.timeoutMs : DEFAULT_API_TIMEOUT_MS;
+    this.logger = logger;
+  }
+
+  getDimensions(): number {
+    return this.dims;
+  }
+
+  getProviderInfo(): EmbeddingProviderInfo {
+    return { provider: "zeroentropy", model: this.model };
+  }
+
+  /** Remote embedding is always ready (stateless HTTP). */
+  isReady(): boolean {
+    return true;
+  }
+
+  /** No-op for remote embedding (no local model to warm up). */
+  startWarmup(): void {
+    // nothing to do — remote API is stateless
+  }
+
+  async embed(text: string, options?: EmbeddingCallOptions): Promise<Float32Array> {
+    const [result] = await this.embedBatch([text], options);
+    return result;
+  }
+
+  async embedBatch(texts: string[], options?: EmbeddingCallOptions): Promise<Float32Array[]> {
+    if (texts.length === 0) return [];
+
+    const processedTexts = truncateEmbeddingInputs(texts, this.maxInputChars, this.logger);
+
+    if (processedTexts.length > MAX_BATCH_SIZE) {
+      const results: Float32Array[] = [];
+      for (let i = 0; i < processedTexts.length; i += MAX_BATCH_SIZE) {
+        const chunk = processedTexts.slice(i, i + MAX_BATCH_SIZE);
+        const chunkResults = await this._callApi(chunk, options?.timeoutMs);
+        results.push(...chunkResults);
+      }
+      return results;
+    }
+
+    return this._callApi(processedTexts, options?.timeoutMs);
+  }
+
+  private async _callApi(texts: string[], timeoutOverride?: number): Promise<Float32Array[]> {
+    // ZeroEntropy rejects requests without `input_type`. We default to
+    // "query" because the recall hot path is the only caller of embed()
+    // that returns a Float32Array; capture-side batches eventually feed
+    // the same vector store, and ZeroEntropy's symmetry between "query"
+    // and "document" makes a single type safe across both directions.
+    const body: Record<string, unknown> = {
+      input: texts,
+      model: this.model,
+      input_type: "query",
+    };
+    if (this.sendDimensions) {
+      // ZeroEntropy's docs list `dimensions` as optional. For zembed-1 the
+      // accepted set is [2560, 1280, 640, 320, 160, 80, 40] (Matryoshka);
+      // any other value is rejected server-side. We forward the user's
+      // configured value verbatim — clamping silently would surprise users
+      // who deliberately picked a smaller dim for storage savings.
+      body.dimensions = this.dims;
+    }
+
+    const fetchUrl = `${this.baseUrl}/models/embed`;
+    const headers: Record<string, string> = {
+      "Content-Type": "application/json",
+      Authorization: `Bearer ${this.apiKey}`,
+    };
+
+    const json = (await postEmbeddingRequest({
+      fetchUrl,
+      headers,
+      body,
+      timeoutMs: timeoutOverride ?? this.timeoutMs,
+    })) as ZeroEntropyEmbeddingResponse;
+
+    if (!json.results || !Array.isArray(json.results)) {
+      throw new Error("ZeroEntropy embedding API returned unexpected format: missing 'results' array");
+    }
+    // ZeroEntropy preserves input order via array position (no `index` field).
+    return json.results.map((r) => sanitizeAndNormalize(r.embedding));
   }
 }
 
@@ -597,6 +790,12 @@ export function createEmbeddingService(
   config: EmbeddingConfig | undefined,
   logger?: Logger,
 ): EmbeddingService {
+  // ZeroEntropy speaks a non-OpenAI wire format and has its own service class.
+  if (config && config.provider === "zeroentropy" && "apiKey" in config && config.apiKey) {
+    logger?.debug?.(`${TAG} Using ZeroEntropy embedding (model=${config.model})`);
+    return new ZeroEntropyEmbeddingService(config as OpenAIEmbeddingConfig, logger);
+  }
+
   // Remote OpenAI-compatible provider: any provider value other than "local"
   if (config && config.provider !== "local" && "apiKey" in config && config.apiKey) {
     logger?.debug?.(`${TAG} Using remote embedding (provider=${config.provider}, model=${config.model})`);