> Evaluated against `openai/openai-agents-python` v0.14.6 (`/tmp/openai-agents`). Maps every Strix subsystem from `HARNESS_WIKI.md` (Strix at `9fb1012`) onto SDK primitives.
>
> **Revision 2** — incorporates: (a) confirmed multi-agent + messaging is bridgeable via `call_model_input_filter`; (b) accepted tradeoffs on XML tool format, skills-as-tool-output, sandbox subclass; (c) tool-execution threading & timeout deltas (sequential→parallel, no default timeouts, no auto sync offload).
10. [Migration plan (step-by-step)](#10-migration-plan-step-by-step)
11. [Risks & open questions](#11-risks--open-questions)
---
## 1. TL;DR
**Verdict: full migration is feasible.** ~25–35 engineer-days for full parity, including parallel multi-agent + messaging, with three accepted tradeoffs and one custom Docker-client subclass.
| Concern | Original status | Revised status |
|---|---|---|
| Concurrent multi-agent graph | "Critical / not bridgeable" | **Bridgeable.**`call_model_input_filter` + `asyncio.create_task` + shared `Session` + a `MessageBus` we own. Architecturally identical to today's `_agent_messages` injection in `_check_agent_messages`. ~400 LOC, full parity (true parallel, peer-to-peer messaging, wait_for_message, view_agent_graph, identity injection, stat aggregation). |
| XML tool-call format | "Critical" | **Accepted tradeoff.** SDK is JSON-native (provider-native via LiteLLM extension for non-OpenAI). No real loss — provider-native tool use is cleaner. Multi-provider survives. |
| `load_skill` mid-run prompt mutation | "High loss" | **Accepted tradeoff.** Skills returned as tool output; model sees them in conversation history. Slightly more memory-compressor-eviction-prone, but cleaner semantics. |
| Sandbox `cap_add` / `extra_hosts` | "High" | **Solvable.** Subclass `DockerSandboxClient` and inject the kwargs. ~50–80 LOC. |
| Tool execution semantics | not addressed | **Net upgrade.** SDK runs tool calls in **parallel** within a turn (Strix is sequential). No default per-tool timeout (Strix has 120s) — we add a `strix_tool()` factory to re-impose defaults. No auto sync→thread offload (Strix's tool server `asyncio.to_thread`s every call) — we wrap sync code ourselves. |
**No remaining showstoppers.** All gaps now have concrete bridges.
| Streaming early-truncate at `</function>` | `result.cancel(mode="after_turn")` (turn-level only) | Partial | Lose token savings on over-generating models. ~50–100 LOC custom Model wrapper if we want it back. |
| `view_agent_graph` text rendering | Bus traversal helper | 1:1 | Ours, ~30 LOC. |
| Subagent identity injection (`<agent_delegation>` XML) | Set `agent_id`/`parent_id`/`agent_name` in `RunContextWrapper.context`; child instructions are a callable that pulls from context | 1:1 | Same effect, no XML. |
| Cancellation (`cancel_current_execution`) | `task.cancel()` on the `asyncio.Task` we own (one per agent in the bus) | 1:1 | Identical primitive. |
| Interactive "waiting state" with timeout | `wait_for_message` tool polls bus inbox via `asyncio.sleep` | 1:1 | Same semantics, ~20 LOC. |
| Subagent stat aggregation | `RunHooks.on_llm_end` pushes usage to bus; `on_agent_end` finalizes | 1:1 | Cleaner than today's `_completed_agent_llm_totals` lock-protected dict. |
| Lifecycle hooks (implicit today) | `RunHooks` + `AgentHooks` (7 hooks) | **Gain** | Use these to wire tracer + stats. |
| Vulnerability dedup (separate LLM call) | Function tool that calls a nested `Runner.run` or direct LiteLLM | 1:1 — port as-is. |
| Custom Jinja system prompt | `Agent.instructions: str | Callable[..., str]` | 1:1 — pre-render Jinja before agent creation, or pass an async callable. |
### 3.3 Tool system (Strix §7)
All 13 Strix tools port. Multi-agent-graph tools are now in §4.
| Strix tool | SDK primitive | Effort |
|---|---|---|
| `@register_tool` w/ env-conditional registration | `@function_tool` + per-agent `tools=[...]` list assembled via env checks at agent build time | Low |
| Local-vs-sandbox dispatch | All tools are `@function_tool` async. Sandbox tools are wrappers that POST to our existing FastAPI tool server. **Network isolation + Bearer auth survive at the transport layer.** | Medium |
| Result XML wrap + 10KB head/tail truncation + screenshot extraction | `ToolOutputText` / `ToolOutputImage` / `ToolOutputFileContent`; truncation logic in our wrapper | Low–Medium |
| Sequential tool execution | **SDK runs tool calls in parallel within a turn** (see §5). Net gain. Verify our stateful tools are reentrant-safe (browser singleton already is via its `threading.Lock`). | n/a |
| `cap_add=NET_ADMIN,NET_RAW` + `extra_hosts=host.docker.internal` | **Subclass `DockerSandboxClient`, inject into `containers.create()` kwargs** | Bridgeable | ~80 LOC |
| Caido HTTPS proxy + CA cert + system-wide proxy env | Image-baked (Dockerfile + entrypoint stay as-is); `Manifest.environment` for runtime overrides; custom `CaidoCapability` for the 7 Caido tools + system-prompt instruction block | Bridgeable | ~200 LOC capability |
| FastAPI tool server + Bearer auth | **Stays in the image.** Function tools wrap HTTP calls to it. Network isolation + Bearer auth preserved at transport layer. SDK's "in-process tools" model becomes "function tool that POSTs to localhost:48081 inside our shared session." | 1:1 in effect |
| One container per scan, shared by all agents | `SandboxRunConfig(session=shared_session)` passed into every `Runner.run` call | 1:1 |
| Random host port allocation | We pre-allocate via `socket.bind(0)` and pass to `DockerSandboxClientOptions(exposed_ports=...)` | 1:1 |
| Healthcheck polling | External loop after `client.create()`, polling `session.exec("curl -fs localhost:48081/health")` | Bridgeable | ~30 LOC |
| Container reuse keyed by scan_id | We track our own session map | 1:1 |
| Local source tar-pipe to `/workspace` | `Manifest.entries={"sources": LocalDir(src=Path)}` | 1:1+ — SDK is a strict superset (LocalDir, LocalFile, GitRepo, S3Mount, …) |
| Multi-agent silo via `agent_id` ContextVar | `RunContextWrapper.context["agent_id"]` extracted in stateful tools | 1:1 (explicit instead of implicit) |
| Cleanup via async `docker rm -f` | `await client.delete(session)` wrapped in `try/finally` | 1:1 |
This was the contested section in the previous evaluation. **It's bridgeable, the bridge is small, and the architecture is identical to today's Strix in shape — just lives in our code on top of SDK primitives.**
This filter runs **before every model call** (`run_internal/turn_preparation.py:55-80`). It receives the input list + instructions and returns a (possibly mutated) `ModelInputData(input=[...], instructions=...)`. **This is the exact injection point Strix uses today** in `_check_agent_messages` at the top of every iteration. It's the missing piece.
| Daemon-thread subagent (`threading.Thread`) | `asyncio.create_task(Runner.run(...))` | **Yes in effect.** LLM calls are I/O-bound; both designs get the same effective concurrency. We were never CPU-bound at the agent level. |
| Shared `/workspace` Kali sandbox | Shared `SandboxRunConfig(session=...)` passed to every child's `RunConfig` | Yes |
| Per-iteration message check (`_check_agent_messages` at top of `agent_loop`) | `call_model_input_filter` runs before every LLM call (SDK guarantees this in `turn_preparation.py:55-80`) | Yes |
| `<inter_agent_message>` XML wrap | Filter formats as user-role items with same XML envelope | Yes |
| `wait_for_message` tool with timeout | Tool that polls `bus.inboxes[me]` in `asyncio.sleep` loop | Yes |
| `view_agent_graph` text output | Bus traversal helper | Yes |
| Identity injection via `<agent_delegation>` XML | Set identity in `RunContextWrapper.context`; agent's instructions are a callable that pulls from context | Equivalent (no XML wrapping; identity still flows) |
| Stop on parent | Walk descendants via `parent_of`, cancel each task | Yes (same as Strix today) |
### 4.8 What this design does NOT lose
- **True concurrency** at the LLM-I/O boundary. (Python threading was never giving us CPU parallelism for our workload anyway.)
- **Shared sandbox** semantics — same Kali container, same `/workspace`, same Caido capture, same proxy state.
- **Cross-sibling messaging** — fully bridged via the bus + filter.
- **Stat aggregation** — cleaner via hooks.
- **Per-agent state silo** for stateful tools (browser, terminal, python) — `RunContextWrapper.context["agent_id"]` is the explicit equivalent of the implicit `current_agent_id` ContextVar.
### 4.9 What this design does lose (small)
- **Per-agent task slot serialization** (Strix's tool server cancels a previous in-flight tool when a new one for the same agent arrives). Not actually needed under the SDK because each agent's run loop only emits a new tool call after the previous resolves.
- **Implicit ContextVar magic** — became explicit `ctx.context["agent_id"]` extraction. ~3 LOC helper makes it ergonomic.
---
## 5. Tool execution semantics — what changes
This is the operational gotcha most likely to surprise during migration. Source-verified from `tool_execution.py` and `tool_server.py` (Strix), `run_internal/tool_execution.py` and `tool.py` (SDK).
### 5.1 Side-by-side
| Dimension | Strix | OpenAI SDK |
|---|---|---|
| **Tool calls within one model turn** | **Sequential** (`for inv in invocations` at `executor.py:324`) | **Parallel** (`asyncio.create_task` per call, drained via `asyncio.wait FIRST_COMPLETED` at `tool_execution.py:1412-1430`) |
| **Default per-tool timeout** | 120s (`STRIX_SANDBOX_EXECUTION_TIMEOUT`) + 30s host buffer = 150s outer | **None.** Must opt in via `@function_tool(timeout=N)` |
| **Local/host-side tool timeout** | None — runs in main loop | None unless `timeout_seconds` is set |
| **Sandbox/remote tool timeout** | 120s `asyncio.wait_for` server-side + 150s httpx outer client-side | N/A — SDK has no remote tool concept; we wrap HTTP in a function tool and set timeout ourselves |
| **Connect timeout** | 10s for httpx → sandbox | None built-in — pass `httpx.Timeout(connect=10)` in our tool body |
| **Sync function offload** | Tool server: `asyncio.to_thread(tool_func, ...)` always (`tool_server.py:83`) | **No auto-offload.** Sync code blocks the loop unless we wrap with `asyncio.to_thread` ourselves |
| **Per-agent serialization** | Yes — `agent_tasks[agent_id]`; new request cancels previous (`tool_server.py:94-97`) | No — concurrent calls allowed; not needed anyway since SDK only emits next tool after current resolves |
| **One-failure-cancels-siblings** | N/A (sequential) | `isolate_parallel_failures=True` by default for multi-call turns (`tool_execution.py:1370`) |
| **Cancellation primitive** | `task.cancel()` on host; SIGTERM cancels all server tasks | `asyncio.shield(invoke_task)` (`tool_execution.py:1766`) + outer cancellation; `result.cancel()` for whole-run |
| **Timeout error format** | Returns `"Tool timed out after 120s"` string to the LLM | `default_tool_timeout_error_message(...)` string (or `ToolTimeoutError` if `timeout_behavior="raise_exception"`) |
| **Stateful-tool threading (browser)** | Dedicated daemon thread + own event loop, lock-serialized (`browser_instance.py:34-48`) | Whatever we build inside the tool function (we keep our existing approach) |
### 5.2 Migration implications
1.**Parallel tool calls become a feature.** Strix is sequential; SDK runs them concurrently. For the model emitting `terminal_execute("nmap ...")` + `web_search("CVE-X")` in one turn, this is faster. We verify reentrancy on:
r = await client.post(..., timeout=_SANDBOX_TIMEOUT)
return r.json()
```
5. **Tool error formatting** — set a default `failure_error_function` on `RunConfig` to keep our existing `<tool_result><error>...</error></tool_result>` shape if we want it; otherwise the SDK's default error string is acceptable.
---
## 6. Sandbox bridge
### 6.1 Custom DockerSandboxClient (~80 LOC)
The SDK's `DockerSandboxClient.create()` doesn't expose `cap_add` or `extra_hosts`. Subclass and inject:
```python
# strix/runtime/strix_docker_client.py
from agents.sandbox.sandboxes.docker import DockerSandboxClient
class StrixDockerSandboxClient(DockerSandboxClient):
"""Adds NET_ADMIN, NET_RAW capabilities and host.docker.internal mapping
needed for raw-socket pentest tools and host-served-app testing."""
return "<caido_proxy>All HTTP/HTTPS traffic in this sandbox is captured by Caido. ...</caido_proxy>"
```
### 6.3 Tool server stays put
The FastAPI tool server keeps running on `:48081` inside the container. Each Strix tool becomes an `@strix_tool` that POSTs to it with our existing Bearer token. **Network isolation, Bearer auth, and the entire image build pipeline are unchanged.** What changes is only the host-side dispatcher: instead of `tools/executor.py`, it's `function_tool` bodies that call the same endpoint.
---
## 7. What we still lose control over
Smaller list than before. All accepted as tradeoffs.
1. **Streaming early-truncate at `</function>`.** Token waste on over-generating models. Custom Model wrapper if Bedrock economics matter; otherwise live with it.
2. **Per-chunk streaming timeout** (the Bedrock `60abc09` fix). Same answer — wrap if Bedrock matters.
3. **Mid-run system prompt mutation (`load_skill`).** Skills become tool outputs (model sees them in conversation history). Slightly more memory-compressor-eviction-prone. Acceptable.
4. **Anthropic prompt cache auto-injection.** Becomes per-agent manual `extra_body` setting via a small `make_anthropic_settings()` helper.
5. **Cost tracking.** SDK tracks tokens, not cost. Wire `litellm.completion_cost` in `on_llm_end` hook (~20 LOC).
6. **Vision-less model image stripping.** No automatic fallback. Wrap as Model subclass if non-vision providers matter.
7. **Identity injection in delegation** moves from XML to context dict. Equivalent — no real loss.
---
## 8. What we gain
Net upgrades from the SDK. Things we don't have today:
| Gain | Detail |
|---|---|
| **Sessions / persistence** | 8+ backends (`SQLiteSession`, `RedisSession`, `SQLAlchemySession`, `MongoDBSession`, `DaprSession`, `EncryptedSession`, `OpenAIConversationsSession`, `OpenAIResponsesCompactionSession`). `RunState.to_json()` resumable runs. We currently have nothing. |
5. **End-to-end test**: root spawns 2 children in parallel, children exchange messages, both finish, root aggregates stats. Compare to today's baseline.
2. Multi-agent: parallel children + messaging + cancel.
3. Bedrock + Anthropic + OpenAI parity test.
4. Memory compression at 90K tokens.
5. PII redaction in traces.
6. Run an existing pentest end-to-end and diff outputs against the Strix baseline.
---
## 11. Risks & open questions
1. **`CallModelInputFilter` re-runs on every model call.** If we drain the inbox in the filter and the model call retries (e.g. retryable HTTP error), do we lose messages? Need to verify SDK retry behavior — does `call_model_input_filter` re-run on retry, or does the filtered input get cached for the retry? **Action: read `run_internal/turn_preparation.py:55-80` + retry path before Phase 3.** If messages would be lost, the fix is to drain into a per-call buffer that only commits on successful response.
2. **`Session` semantics under parallel children.** When children share the same `Session` for sandbox state, do their LLM histories cross-contaminate? Children should use distinct logical sessions for history (per-agent) but share the sandbox session. **Action: verify `Session` and `SandboxRunConfig.session` are independent — they are, but write a test.**
3. **`isolate_parallel_failures=True` default.** When the model emits multiple tool calls in one turn and one fails, all siblings get cancelled. We may want `False` for our use case (a failed `nmap` shouldn't kill an in-flight `web_search`). **Action: configure per `RunConfig` once we see real behavior.**
4. **Sandbox tool concurrency under parallel calls.** Today's tool server has per-agent task slot serialization (one tool in flight per agent). Under SDK's parallel-tool-calls model, we'd issue multiple POSTs concurrently for the same agent. Tool server's current behavior is to **cancel the previous task** (`tool_server.py:94-97`), which would break us. **Action: relax tool server to allow concurrent same-agent tool calls, OR set `parallel_tool_calls=False` on `ModelSettings` and stay sequential.** The latter is the safer migration default; revisit later.
5. **Bedrock per-chunk-timeout regression.** Without the custom Model wrapper, Bedrock TCP-stalls return as a class of failure. **Action: decide whether Bedrock matters enough to invest the 1.5 days. If it does, build the wrapper in Phase 1.**
6. **Streaming early-exit at `</function>` cost.** Wasted tokens on every multi-turn for over-generating models. Quantify against a representative scan; if cost delta is small, skip the wrapper.
7. **Memory-compressor eviction risk for tool-output skills.** When `load_skill` returns content as tool output, the compressor may summarize the skill content into oblivion after 15+ messages. **Action: tag skill-load tool outputs in the conversation and configure the compressor to preserve them.**
---
*Revision 2 — incorporates: (a) `call_model_input_filter`-based multi-agent bridge; (b) accepted tradeoffs on XML / skills / sandbox subclass; (c) tool execution semantic deltas (parallel by default, no default timeouts, no auto-offload).*
