refactor(notes): drop disk persistence + shared-wiki prose

The notes tool no longer touches disk. ``_notes_storage`` lives in
memory for the lifetime of one scan process, shared across every
agent in that process via the existing RLock. Process exit clears
the lot — no notes.jsonl event log, no wiki/<slug>.md Markdown
rendering, no replay-on-startup hydration.

Removed ~10 internal helpers (``_get_run_dir``,
``_get_notes_jsonl_path``, ``_append_note_event``,
``_load_notes_from_jsonl``, ``_ensure_notes_loaded``,
``_persist_wiki_note``, ``_remove_wiki_note``,
``_get_wiki_directory``, ``_get_wiki_note_path``,
``_sanitize_wiki_title``) plus the ``_loaded_notes_run_dir`` module
state, ``wiki_filename`` per-note field, and the ``OSError`` branches
that only existed for the wiki write path.

The ``wiki`` category is preserved as a free-form long-form bucket;
it just no longer has any special persistence behaviour.

Skill prompts scrubbed of every "shared wiki memory" / "repo wiki" /
"append a delta before agent_finish" instruction:
``coordination/source_aware_whitebox.md``,
``custom/source_aware_sast.md``,
``scan_modes/{quick,standard,deep}.md``, plus the WHITE-BOX TESTING
block in ``agents/prompts/system_prompt.jinja``.

HARNESS_WIKI.md updated to drop the wiki-as-shared-knowledge-base
description, the per-run output-tree references to ``notes/notes.jsonl``
and ``wiki/{note_id}-{slug}.md``, and the ``is_whitebox`` toggle prose.

Net: -178 LoC in notes/tools.py, -45 LoC across skills/system_prompt
and the wiki doc. The notes tool surface (5 ``@function_tool``s) is
unchanged for the agent.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
0xallam
2026-04-25 23:56:22 -07:00
parent f8213452ea
commit 81703e286f
8 changed files with 20 additions and 263 deletions
+5 -9
View File
@@ -211,7 +211,7 @@ Snapshots (`state.model_dump()`) are stored verbatim in the agent-graph node whe
**Identity injection**: parent task is wrapped in `<agent_delegation>...<agent_identity>` so the child knows its name/ID and is told to never echo it (`:238-266`). **Identity injection**: parent task is wrapped in `<agent_delegation>...<agent_identity>` so the child knows its name/ID and is told to never echo it (`:238-266`).
**Finish from subagent** (`agent_finish`, `:567-685`): only callable when `parent_id != None`. Builds `<agent_completion_report>` XML with summary/findings/recommendations and pushes it onto the parent's inbox. If the agent is whitebox, the wiki note is updated with a delta. **Finish from subagent** (`agent_finish`, `:567-685`): only callable when `parent_id != None`. Builds `<agent_completion_report>` XML with summary/findings/recommendations and pushes it onto the parent's inbox.
**Finish from root** (`finish_scan`, `tools/finish/finish_actions.py:86-149`): only callable from root (`parent_id is None`); blocks if any sibling/child agent is still `running`/`stopping`. Triggers `tracer.update_scan_final_fields()` which writes `penetration_test_report.md`. **Finish from root** (`finish_scan`, `tools/finish/finish_actions.py:86-149`): only callable from root (`parent_id is None`); blocks if any sibling/child agent is still `running`/`stopping`. Triggers `tracer.update_scan_final_fields()` which writes `penetration_test_report.md`.
@@ -382,13 +382,11 @@ Supports HTTPQL filter syntax for request queries. Pagination (`offset`, `limit`
Hardcoded port `48080`. Caido v0.56.0 pinned in `containers/Dockerfile` (override via `--build-arg CAIDO_VERSION=...`). Hardcoded port `48080`. Caido v0.56.0 pinned in `containers/Dockerfile` (override via `--build-arg CAIDO_VERSION=...`).
#### Notes (`strix/tools/notes/`) — `create_note`, `list_notes`, `get_note`, `update_note`, `delete_note` (+ internal `append_note_content`) #### Notes (`strix/tools/notes/`) — `create_note`, `list_notes`, `get_note`, `update_note`, `delete_note`
In-memory dict + JSONL persistence at `{run_dir}/notes/notes.jsonl`. Wiki-category notes additionally rendered as Markdown to `{run_dir}/wiki/{note_id}-{title}.md` so they're human-readable artifacts of the run. Pure in-memory dict, shared across every agent in the same scan for the lifetime of the process. **Not persisted** — process exit clears the lot.
Categories: `general | findings | methodology | questions | plan | wiki`. IDs are 5-char UUID hex (collision-retry up to 20 attempts). Thread-safe via RLock. List preview defaults to 280 chars per note. Categories: `general | findings | methodology | questions | plan | wiki`. IDs are 5-char UUID hex (collision-retry up to 20 attempts). Thread-safe via RLock. List preview defaults to 280 chars per note.
The wiki note in particular is **the shared whitebox knowledge base** between root and subagents; `agent_finish` for whitebox subagents auto-appends a delta (see §5.3).
#### Todos (`strix/tools/todo/`) — 6 tools #### Todos (`strix/tools/todo/`) — 6 tools
Per-agent in-memory storage; **not persisted**. Priorities `critical | high | normal | low`; statuses `pending | in_progress | done`. Bulk create/update via JSON list. IDs are 6-char UUID hex. Per-agent in-memory storage; **not persisted**. Priorities `critical | high | normal | low`; statuses `pending | in_progress | done`. Bulk create/update via JSON list. IDs are 6-char UUID hex.
@@ -536,8 +534,6 @@ Created and managed by `telemetry/tracer.py`. Contents:
- `events.jsonl` — every span/event in append-only JSONL (thread-safe writes). - `events.jsonl` — every span/event in append-only JSONL (thread-safe writes).
- `vulnerabilities/vuln_{id}.json` — one file per finding, sorted by severity, dedup-checked. - `vulnerabilities/vuln_{id}.json` — one file per finding, sorted by severity, dedup-checked.
- `penetration_test_report.md` — final markdown report (executive summary + methodology + technical analysis + recommendations). - `penetration_test_report.md` — final markdown report (executive summary + methodology + technical analysis + recommendations).
- `notes/notes.jsonl` — note ops audit log.
- `wiki/{note_id}-{slug}.md` — human-readable wiki notes.
- `<target_subdir>/` — local source clones, per-target. - `<target_subdir>/` — local source clones, per-target.
There is **no execution checkpointing** — if the process crashes mid-run, the agent restarts from scratch on retry. Resumability is limited to the interactive-mode wait/resume on inbound messages. There is **no execution checkpointing** — if the process crashes mid-run, the agent restarts from scratch on retry. Resumability is limited to the interactive-mode wait/resume on inbound messages.
@@ -576,7 +572,7 @@ There **is no separate persona file**. All agents are `StrixAgent` instances usi
- `parent_id` (root agent vs subagent → different finish tools, different prompts injected). - `parent_id` (root agent vs subagent → different finish tools, different prompts injected).
- Loaded skills (root gets `root_agent`; whitebox gets `coordination/source_aware_whitebox` + `custom/source_aware_sast`). - Loaded skills (root gets `root_agent`; whitebox gets `coordination/source_aware_whitebox` + `custom/source_aware_sast`).
- `system_prompt_context.authorized_targets` (only set on root). - `system_prompt_context.authorized_targets` (only set on root).
- `is_whitebox` flag (toggles wiki-note auto-update on subagent finish). - `is_whitebox` flag (selects the whitebox skill stack).
--- ---
@@ -754,7 +750,7 @@ Disabled by `STRIX_POSTHOG_TELEMETRY=0` or `STRIX_TELEMETRY=0`.
| `strix/tools/terminal/` | tmux/libtmux tool. | | `strix/tools/terminal/` | tmux/libtmux tool. |
| `strix/tools/python/` | IPython tool. | | `strix/tools/python/` | IPython tool. |
| `strix/tools/proxy/` | Caido GraphQL client. | | `strix/tools/proxy/` | Caido GraphQL client. |
| `strix/tools/notes/` | Notes + wiki. | | `strix/tools/notes/` | In-memory notes (shared across agents in a run). |
| `strix/tools/todo/` | In-memory todos. | | `strix/tools/todo/` | In-memory todos. |
| `strix/tools/reporting/` | Vulnerability reports + CVSS. | | `strix/tools/reporting/` | Vulnerability reports + CVSS. |
| `strix/tools/web_search/` | Perplexity. | | `strix/tools/web_search/` | Perplexity. |
+2 -4
View File
@@ -112,11 +112,9 @@ WHITE-BOX TESTING (code provided):
- MUST perform BOTH static AND dynamic analysis - MUST perform BOTH static AND dynamic analysis
- Static: Use source-aware triage first to map risk quickly (`semgrep`, `ast-grep`, Tree-sitter tooling, `gitleaks`, `trufflehog`, `trivy fs`). Then review code for vulnerabilities - Static: Use source-aware triage first to map risk quickly (`semgrep`, `ast-grep`, Tree-sitter tooling, `gitleaks`, `trufflehog`, `trivy fs`). Then review code for vulnerabilities
- Static coverage floor: execute at least one structural AST mapping pass (`sg` and/or Tree-sitter) per repository and keep artifact output - Static coverage floor: execute at least one structural AST mapping pass (`sg` and/or Tree-sitter) per repository and keep artifact output
- Static coverage target per repository: run one `semgrep` pass, one secrets pass (`gitleaks` and/or `trufflehog`), one `trivy fs` pass, and one AST-structural pass (`sg` and/or Tree-sitter); if any are skipped, record why in the shared wiki - Static coverage target per repository: run one `semgrep` pass, one secrets pass (`gitleaks` and/or `trufflehog`), one `trivy fs` pass, and one AST-structural pass (`sg` and/or Tree-sitter)
- Keep AST artifacts bounded and high-signal: scope to relevant paths/hypotheses, avoid whole-repo generic function dumps - Keep AST artifacts bounded and high-signal: scope to relevant paths/hypotheses, avoid whole-repo generic function dumps
- AST target selection rule: build `sg-targets.txt` from `semgrep.json` scope first (`paths.scanned`, fallback to unique `results[].path`), then run `xargs ... sg run` against that file list. Only use path-heuristic fallback if semgrep scope is unavailable, and log fallback reason in the wiki. - AST target selection rule: build `sg-targets.txt` from `semgrep.json` scope first (`paths.scanned`, fallback to unique `results[].path`), then run `xargs ... sg run` against that file list. Only use path-heuristic fallback if semgrep scope is unavailable.
- Shared memory: Use notes as shared working memory; discover wiki notes with `list_notes`, then read the selected one via `get_note(note_id=...)` before analysis
- Before `agent_finish`/`finish_scan`, update the shared repo wiki with scanner summaries, key routes/sinks, and dynamic follow-up plan
- Dynamic: Run the application and test live to validate exploitability - Dynamic: Run the application and test live to validate exploitability
- NEVER rely solely on static code analysis when dynamic validation is possible - NEVER rely solely on static code analysis when dynamic validation is possible
- Begin with fast source triage and dynamic run preparation in parallel; use static findings to prioritize live testing. - Begin with fast source triage and dynamic run preparation in parallel; use static findings to prioritize live testing.
@@ -15,11 +15,10 @@ Increase white-box coverage by combining source-aware triage with dynamic valida
1. Build a quick source map before deep exploitation, including at least one AST-structural pass (`sg` or `tree-sitter`) scoped to relevant paths. 1. Build a quick source map before deep exploitation, including at least one AST-structural pass (`sg` or `tree-sitter`) scoped to relevant paths.
- For `sg` baseline, derive `sg-targets.txt` from `semgrep.json` scope first (`paths.scanned`, fallback to unique `results[].path`) and run `xargs ... sg run` on that list. - For `sg` baseline, derive `sg-targets.txt` from `semgrep.json` scope first (`paths.scanned`, fallback to unique `results[].path`) and run `xargs ... sg run` on that list.
- Only fall back to path heuristics when semgrep scope is unavailable, and record the fallback reason in the repo wiki. - Only fall back to path heuristics when semgrep scope is unavailable.
2. Run first-pass static triage to rank high-risk paths. 2. Run first-pass static triage to rank high-risk paths.
3. Use triage outputs to prioritize dynamic PoC validation. 3. Use triage outputs to prioritize dynamic PoC validation.
4. Keep findings evidence-driven: no report without validation. 4. Keep findings evidence-driven: no report without validation.
5. Keep shared wiki memory current so all agents can reuse context.
## Source-Aware Triage Stack ## Source-Aware Triage Stack
@@ -34,7 +33,6 @@ Coverage target per repository:
- one AST structural pass (`sg` and/or `tree-sitter`) - one AST structural pass (`sg` and/or `tree-sitter`)
- one secrets pass (`gitleaks` and/or `trufflehog`) - one secrets pass (`gitleaks` and/or `trufflehog`)
- one `trivy fs` pass - one `trivy fs` pass
- if any part is skipped, log the reason in the shared wiki note
## Agent Delegation Guidance ## Agent Delegation Guidance
@@ -42,25 +40,6 @@ Coverage target per repository:
- For source-heavy subtasks, prefer creating child agents with `source_aware_sast` skill. - For source-heavy subtasks, prefer creating child agents with `source_aware_sast` skill.
- Use source findings to shape payloads and endpoint selection for dynamic testing. - Use source findings to shape payloads and endpoint selection for dynamic testing.
## Wiki Note Requirement (Source Map)
When source is present, maintain one wiki note per repository and keep it current.
Operational rules:
- At task start, call `list_notes` with `category=wiki`, then read the selected wiki with `get_note(note_id=...)`.
- If no repo wiki exists, create one with `create_note` and `category=wiki`.
- Update the same wiki via `update_note`; avoid creating duplicate wiki notes for the same repo.
- Child agents should read wiki notes first via `get_note`, then extend with new evidence from their scope.
- Before calling `agent_finish`, each source-focused child agent should append a short delta update to the shared repo wiki (scanner outputs, route/sink map deltas, dynamic follow-ups).
Recommended sections:
- Architecture overview
- Entrypoints and routing
- AuthN/AuthZ model
- High-risk sinks and trust boundaries
- Static scanner summary
- Dynamic validation follow-ups
## Validation Guardrails ## Validation Guardrails
- Static findings are hypotheses until validated. - Static findings are hypotheses until validated.
-32
View File
@@ -15,17 +15,6 @@ Run tools from repo root and store outputs in a dedicated artifact directory:
mkdir -p /workspace/.strix-source-aware mkdir -p /workspace/.strix-source-aware
``` ```
Before scanning, check shared wiki memory:
```text
1) list_notes(category="wiki")
2) get_note(note_id=...) for the selected repo wiki before analysis
3) Reuse matching repo wiki note if present
4) create_note(category="wiki") only if missing
```
After every major source-analysis batch, update the same repo wiki note with `update_note` so other agents can reuse your latest map.
## Baseline Coverage Bundle (Recommended) ## Baseline Coverage Bundle (Recommended)
Run this baseline once per repository before deep narrowing: Run this baseline once per repository before deep narrowing:
@@ -74,8 +63,6 @@ trivy fs --scanners vuln,misconfig --timeout 30m --offline-scan \
--format json --output "$ART/trivy-fs.json" . || true --format json --output "$ART/trivy-fs.json" . || true
``` ```
If one tool is skipped or fails, record that in the shared wiki note along with the reason.
## Semgrep First Pass ## Semgrep First Pass
Use Semgrep as the default static triage pass: Use Semgrep as the default static triage pass:
@@ -141,27 +128,8 @@ trivy fs --scanners vuln,misconfig --timeout 30m --offline-scan \
3. Build dynamic PoCs that reproduce the suspected issue. 3. Build dynamic PoCs that reproduce the suspected issue.
4. Report only after dynamic validation succeeds. 4. Report only after dynamic validation succeeds.
## Wiki Update Template
Keep one wiki note per repository and update these sections:
```text
## Architecture
## Entrypoints
## AuthN/AuthZ
## High-Risk Sinks
## Static Findings Summary
## Dynamic Validation Follow-Ups
```
Before `agent_finish`, make one final `update_note` call to capture:
- scanner artifacts and paths
- top validated/invalidated hypotheses
- concrete dynamic follow-up tasks
## Anti-Patterns ## Anti-Patterns
- Do not treat scanner output as final truth. - Do not treat scanner output as final truth.
- Do not spend full cycles on low-signal pattern matches. - Do not spend full cycles on low-signal pattern matches.
- Do not report source-only findings without validation evidence. - Do not report source-only findings without validation evidence.
- Do not create multiple wiki notes for the same repository when one already exists.
-2
View File
@@ -15,7 +15,6 @@ Thorough understanding before exploitation. Test every parameter, every endpoint
**Whitebox (source available)** **Whitebox (source available)**
- Map every file, module, and code path in the repository - Map every file, module, and code path in the repository
- Load and maintain shared `wiki` notes from the start (`list_notes(category="wiki")` then `get_note(note_id=...)`), then continuously update one repo note
- Start with broad source-aware triage (`semgrep`, `ast-grep`, `gitleaks`, `trufflehog`, `trivy fs`) and use outputs to drive deep review - Start with broad source-aware triage (`semgrep`, `ast-grep`, `gitleaks`, `trufflehog`, `trivy fs`) and use outputs to drive deep review
- Execute at least one structural AST pass (`sg` and/or Tree-sitter) per repository and store artifacts for reuse - Execute at least one structural AST pass (`sg` and/or Tree-sitter) per repository and store artifacts for reuse
- Keep AST artifacts bounded and query-driven (target relevant paths/sinks first; avoid whole-repo generic function dumps) - Keep AST artifacts bounded and query-driven (target relevant paths/sinks first; avoid whole-repo generic function dumps)
@@ -31,7 +30,6 @@ Thorough understanding before exploitation. Test every parameter, every endpoint
- Review file handling: upload, download, processing - Review file handling: upload, download, processing
- Understand the deployment model and infrastructure assumptions - Understand the deployment model and infrastructure assumptions
- Check all dependency versions and repository risks against CVE/misconfiguration data - Check all dependency versions and repository risks against CVE/misconfiguration data
- Before final completion, update the shared repo wiki with scanner summary + dynamic follow-ups
**Blackbox (no source)** **Blackbox (no source)**
- Exhaustive subdomain enumeration with multiple sources and tools - Exhaustive subdomain enumeration with multiple sources and tools
-2
View File
@@ -15,7 +15,6 @@ Optimize for fast feedback on critical security issues. Skip exhaustive enumerat
**Whitebox (source available)** **Whitebox (source available)**
- Focus on recent changes: git diffs, new commits, modified files—these are most likely to contain fresh bugs - Focus on recent changes: git diffs, new commits, modified files—these are most likely to contain fresh bugs
- Read existing `wiki` notes first (`list_notes(category="wiki")` then `get_note(note_id=...)`) to avoid remapping from scratch
- Run a fast static triage on changed files first (`semgrep`, then targeted `sg` queries) - Run a fast static triage on changed files first (`semgrep`, then targeted `sg` queries)
- Run at least one lightweight AST pass (`sg` or Tree-sitter) so structural mapping is not skipped - Run at least one lightweight AST pass (`sg` or Tree-sitter) so structural mapping is not skipped
- Keep AST commands tightly scoped to changed or high-risk paths; avoid broad repository-wide pattern dumps - Keep AST commands tightly scoped to changed or high-risk paths; avoid broad repository-wide pattern dumps
@@ -23,7 +22,6 @@ Optimize for fast feedback on critical security issues. Skip exhaustive enumerat
- Identify security-sensitive patterns in changed code: auth checks, input handling, database queries, file operations - Identify security-sensitive patterns in changed code: auth checks, input handling, database queries, file operations
- Trace user input through modified code paths - Trace user input through modified code paths
- Check if security controls were modified or bypassed - Check if security controls were modified or bypassed
- Before completion, update the shared repo wiki with what changed and what needs dynamic follow-up
**Blackbox (no source)** **Blackbox (no source)**
- Map authentication and critical user flows - Map authentication and critical user flows
-2
View File
@@ -15,7 +15,6 @@ Systematic testing across the full attack surface. Understand the application be
**Whitebox (source available)** **Whitebox (source available)**
- Map codebase structure: modules, entry points, routing - Map codebase structure: modules, entry points, routing
- Start by loading existing `wiki` notes (`list_notes(category="wiki")` then `get_note(note_id=...)`) and update one shared repo note as mapping evolves
- Run `semgrep` first-pass triage to prioritize risky flows before deep manual review - Run `semgrep` first-pass triage to prioritize risky flows before deep manual review
- Run at least one AST-structural mapping pass (`sg` and/or Tree-sitter), then use outputs for route, sink, and trust-boundary mapping - Run at least one AST-structural mapping pass (`sg` and/or Tree-sitter), then use outputs for route, sink, and trust-boundary mapping
- Keep AST output bounded to relevant paths and hypotheses; avoid whole-repo generic function dumps - Keep AST output bounded to relevant paths and hypotheses; avoid whole-repo generic function dumps
@@ -25,7 +24,6 @@ Systematic testing across the full attack surface. Understand the application be
- Analyze database interactions and ORM usage - Analyze database interactions and ORM usage
- Check dependencies and repo risks with `trivy fs`, `gitleaks`, and `trufflehog` - Check dependencies and repo risks with `trivy fs`, `gitleaks`, and `trufflehog`
- Understand the data model and sensitive data locations - Understand the data model and sensitive data locations
- Before completion, update the shared repo wiki with source findings summary and dynamic validation next steps
**Blackbox (no source)** **Blackbox (no source)**
- Crawl application thoroughly, interact with every feature - Crawl application thoroughly, interact with every feature
+12 -190
View File
@@ -1,10 +1,10 @@
"""Per-run notes (shared across agents). """Per-run notes (shared across agents).
Persisted to ``run_dir/notes/notes.jsonl`` (replayable event log) and, Pure in-memory state for the lifetime of one scan: the module-level
for the ``wiki`` category, also rendered as Markdown to ``_notes_storage`` dict is visible to every agent in the same process.
``run_dir/wiki/<slug>.md``. Concurrent appends are serialised by a No disk I/O process exit clears the lot. Concurrent writers are
threading.RLock so two agents writing simultaneously can't corrupt serialised by ``_notes_lock`` since each tool entry-point dispatches
the JSONL. the impl onto a worker thread via ``asyncio.to_thread``.
""" """
from __future__ import annotations from __future__ import annotations
@@ -15,11 +15,7 @@ import logging
import threading import threading
import uuid import uuid
from datetime import UTC, datetime from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any from typing import Any
if TYPE_CHECKING:
from pathlib import Path
from agents import RunContextWrapper, function_tool from agents import RunContextWrapper, function_tool
@@ -30,162 +26,14 @@ logger = logging.getLogger(__name__)
_notes_storage: dict[str, dict[str, Any]] = {} _notes_storage: dict[str, dict[str, Any]] = {}
_VALID_NOTE_CATEGORIES = ["general", "findings", "methodology", "questions", "plan", "wiki"] _VALID_NOTE_CATEGORIES = ["general", "findings", "methodology", "questions", "plan", "wiki"]
_notes_lock = threading.RLock() _notes_lock = threading.RLock()
_loaded_notes_run_dir: str | None = None
_DEFAULT_CONTENT_PREVIEW_CHARS = 280 _DEFAULT_CONTENT_PREVIEW_CHARS = 280
def _get_run_dir() -> Path | None:
try:
from strix.telemetry.tracer import get_global_tracer
tracer = get_global_tracer()
if not tracer:
return None
return tracer.get_run_dir()
except (ImportError, OSError, RuntimeError):
return None
def _get_notes_jsonl_path() -> Path | None:
run_dir = _get_run_dir()
if not run_dir:
return None
notes_dir = run_dir / "notes"
notes_dir.mkdir(parents=True, exist_ok=True)
return notes_dir / "notes.jsonl"
def _append_note_event(op: str, note_id: str, note: dict[str, Any] | None = None) -> None:
"""Append one note operation to the run's ``notes/notes.jsonl``.
C6: hold ``_notes_lock`` across the file open + write so two
concurrent agents can't interleave bytes mid-line.
"""
notes_path = _get_notes_jsonl_path()
if not notes_path:
return
event: dict[str, Any] = {
"timestamp": datetime.now(UTC).isoformat(),
"op": op,
"note_id": note_id,
}
if note is not None:
event["note"] = note
with _notes_lock, notes_path.open("a", encoding="utf-8") as f:
f.write(f"{json.dumps(event, ensure_ascii=True)}\n")
def _load_notes_from_jsonl(notes_path: Path) -> dict[str, dict[str, Any]]:
hydrated: dict[str, dict[str, Any]] = {}
if not notes_path.exists():
return hydrated
with notes_path.open(encoding="utf-8") as f:
for raw_line in f:
line = raw_line.strip()
if not line:
continue
try:
event = json.loads(line)
except json.JSONDecodeError:
continue
op = str(event.get("op", "")).strip().lower()
note_id = str(event.get("note_id", "")).strip()
if not note_id or op not in {"create", "update", "delete"}:
continue
if op == "delete":
hydrated.pop(note_id, None)
continue
note = event.get("note")
if not isinstance(note, dict):
continue
existing = hydrated.get(note_id, {})
existing.update(note)
hydrated[note_id] = existing
return hydrated
def _ensure_notes_loaded() -> None:
global _loaded_notes_run_dir # noqa: PLW0603
run_dir = _get_run_dir()
run_dir_key = str(run_dir.resolve()) if run_dir else "__no_run_dir__"
if _loaded_notes_run_dir == run_dir_key:
return
_notes_storage.clear()
notes_path = _get_notes_jsonl_path()
if notes_path:
_notes_storage.update(_load_notes_from_jsonl(notes_path))
try:
for note_id, note in _notes_storage.items():
if note.get("category") == "wiki":
_persist_wiki_note(note_id, note)
except OSError:
pass
_loaded_notes_run_dir = run_dir_key
def _sanitize_wiki_title(title: str) -> str:
cleaned = "".join(ch.lower() if ch.isalnum() else "-" for ch in title.strip())
slug = "-".join(part for part in cleaned.split("-") if part)
return slug or "wiki-note"
def _get_wiki_directory() -> Path | None:
try:
run_dir = _get_run_dir()
if not run_dir:
return None
wiki_dir = run_dir / "wiki"
wiki_dir.mkdir(parents=True, exist_ok=True)
except OSError:
return None
else:
return wiki_dir
def _get_wiki_note_path(note_id: str, note: dict[str, Any]) -> Path | None:
wiki_dir = _get_wiki_directory()
if not wiki_dir:
return None
wiki_filename = note.get("wiki_filename")
if not isinstance(wiki_filename, str) or not wiki_filename.strip():
title = note.get("title", "wiki-note")
wiki_filename = f"{note_id}-{_sanitize_wiki_title(str(title))}.md"
note["wiki_filename"] = wiki_filename
return wiki_dir / wiki_filename
def _persist_wiki_note(note_id: str, note: dict[str, Any]) -> None:
wiki_path = _get_wiki_note_path(note_id, note)
if not wiki_path:
return
tags = note.get("tags", [])
tags_line = ", ".join(str(tag) for tag in tags) if isinstance(tags, list) and tags else "none"
content = (
f"# {note.get('title', 'Wiki Note')}\n\n"
f"**Note ID:** {note_id}\n"
f"**Created:** {note.get('created_at', '')}\n"
f"**Updated:** {note.get('updated_at', '')}\n"
f"**Tags:** {tags_line}\n\n"
"## Content\n\n"
f"{note.get('content', '')}\n"
)
wiki_path.write_text(content, encoding="utf-8")
def _remove_wiki_note(note_id: str, note: dict[str, Any]) -> None:
wiki_path = _get_wiki_note_path(note_id, note)
if not wiki_path:
return
if wiki_path.exists():
wiki_path.unlink()
def _filter_notes( def _filter_notes(
category: str | None = None, category: str | None = None,
tags: list[str] | None = None, tags: list[str] | None = None,
search_query: str | None = None, search_query: str | None = None,
) -> list[dict[str, Any]]: ) -> list[dict[str, Any]]:
_ensure_notes_loaded()
filtered: list[dict[str, Any]] = [] filtered: list[dict[str, Any]] = []
for note_id, note in _notes_storage.items(): for note_id, note in _notes_storage.items():
if category and note.get("category") != category: if category and note.get("category") != category:
@@ -220,9 +68,6 @@ def _to_note_listing_entry(
"created_at": note.get("created_at", ""), "created_at": note.get("created_at", ""),
"updated_at": note.get("updated_at", ""), "updated_at": note.get("updated_at", ""),
} }
wiki_filename = note.get("wiki_filename")
if isinstance(wiki_filename, str) and wiki_filename:
entry["wiki_filename"] = wiki_filename
content = str(note.get("content", "")) content = str(note.get("content", ""))
if include_content: if include_content:
entry["content"] = content entry["content"] = content
@@ -234,16 +79,14 @@ def _to_note_listing_entry(
return entry return entry
def _create_note_impl( # noqa: PLR0911 def _create_note_impl(
title: str, title: str,
content: str, content: str,
category: str = "general", category: str = "general",
tags: list[str] | None = None, tags: list[str] | None = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
"""Create one note. Public — used by tests."""
with _notes_lock: with _notes_lock:
try: try:
_ensure_notes_loaded()
if not title or not title.strip(): if not title or not title.strip():
return {"success": False, "error": "Title cannot be empty", "note_id": None} return {"success": False, "error": "Title cannot be empty", "note_id": None}
if not content or not content.strip(): if not content or not content.strip():
@@ -276,13 +119,8 @@ def _create_note_impl( # noqa: PLR0911
"updated_at": timestamp, "updated_at": timestamp,
} }
_notes_storage[note_id] = note _notes_storage[note_id] = note
_append_note_event("create", note_id, note)
if category == "wiki":
_persist_wiki_note(note_id, note)
except (ValueError, TypeError) as e: except (ValueError, TypeError) as e:
return {"success": False, "error": f"Failed to create note: {e}", "note_id": None} return {"success": False, "error": f"Failed to create note: {e}", "note_id": None}
except OSError as e:
return {"success": False, "error": f"Failed to persist wiki note: {e}", "note_id": None}
else: else:
return { return {
"success": True, "success": True,
@@ -314,7 +152,6 @@ def _list_notes_impl(
def _get_note_impl(note_id: str) -> dict[str, Any]: def _get_note_impl(note_id: str) -> dict[str, Any]:
with _notes_lock: with _notes_lock:
try: try:
_ensure_notes_loaded()
if not note_id or not note_id.strip(): if not note_id or not note_id.strip():
return {"success": False, "error": "Note ID cannot be empty", "note": None} return {"success": False, "error": "Note ID cannot be empty", "note": None}
note = _notes_storage.get(note_id) note = _notes_storage.get(note_id)
@@ -340,7 +177,6 @@ def _update_note_impl(
) -> dict[str, Any]: ) -> dict[str, Any]:
with _notes_lock: with _notes_lock:
try: try:
_ensure_notes_loaded()
if note_id not in _notes_storage: if note_id not in _notes_storage:
return {"success": False, "error": f"Note with ID '{note_id}' not found"} return {"success": False, "error": f"Note with ID '{note_id}' not found"}
note = _notes_storage[note_id] note = _notes_storage[note_id]
@@ -355,13 +191,8 @@ def _update_note_impl(
if tags is not None: if tags is not None:
note["tags"] = tags note["tags"] = tags
note["updated_at"] = datetime.now(UTC).isoformat() note["updated_at"] = datetime.now(UTC).isoformat()
_append_note_event("update", note_id, note)
if note.get("category") == "wiki":
_persist_wiki_note(note_id, note)
except (ValueError, TypeError) as e: except (ValueError, TypeError) as e:
return {"success": False, "error": f"Failed to update note: {e}"} return {"success": False, "error": f"Failed to update note: {e}"}
except OSError as e:
return {"success": False, "error": f"Failed to persist wiki note: {e}"}
else: else:
return { return {
"success": True, "success": True,
@@ -372,19 +203,13 @@ def _update_note_impl(
def _delete_note_impl(note_id: str) -> dict[str, Any]: def _delete_note_impl(note_id: str) -> dict[str, Any]:
with _notes_lock: with _notes_lock:
try: try:
_ensure_notes_loaded()
if note_id not in _notes_storage: if note_id not in _notes_storage:
return {"success": False, "error": f"Note with ID '{note_id}' not found"} return {"success": False, "error": f"Note with ID '{note_id}' not found"}
note = _notes_storage[note_id] note = _notes_storage[note_id]
note_title = note["title"] note_title = note["title"]
if note.get("category") == "wiki":
_remove_wiki_note(note_id, note)
del _notes_storage[note_id] del _notes_storage[note_id]
_append_note_event("delete", note_id)
except (ValueError, TypeError) as e: except (ValueError, TypeError) as e:
return {"success": False, "error": f"Failed to delete note: {e}"} return {"success": False, "error": f"Failed to delete note: {e}"}
except OSError as e:
return {"success": False, "error": f"Failed to delete wiki note: {e}"}
else: else:
return { return {
"success": True, "success": True,
@@ -405,10 +230,9 @@ async def create_note(
) -> str: ) -> str:
"""Document an observation, finding, methodology step, or research note. """Document an observation, finding, methodology step, or research note.
Notes are your **shared run memory** they're visible to every Notes are visible to every agent in the same scan for the lifetime
agent in the same scan and persist to ``run_dir/notes/notes.jsonl`` of the run; they live in-memory only and are cleared when the
(replayable event log). Wiki-category notes are additionally process exits.
rendered as Markdown under ``run_dir/wiki/<slug>.md``.
For actionable tasks, use ``todo`` instead notes are for capturing For actionable tasks, use ``todo`` instead notes are for capturing
information, todos are for tracking work. information, todos are for tracking work.
@@ -422,9 +246,7 @@ async def create_note(
useful for the final scan report. useful for the final scan report.
- ``questions`` open questions / things to come back to. - ``questions`` open questions / things to come back to.
- ``plan`` multi-step plans you want to track. - ``plan`` multi-step plans you want to track.
- ``wiki`` repository or target source maps shared across agents - ``wiki`` long-form repository or target maps.
in the same run. Use this for codebase architecture notes the
whole agent tree should see.
Tags are free-form (e.g. ``["sqli", "auth", "critical"]``) useful Tags are free-form (e.g. ``["sqli", "auth", "critical"]``) useful
for later ``list_notes(tags=...)`` filtering. for later ``list_notes(tags=...)`` filtering.
@@ -528,7 +350,7 @@ async def update_note(
@function_tool(timeout=30) @function_tool(timeout=30)
async def delete_note(ctx: RunContextWrapper, note_id: str) -> str: async def delete_note(ctx: RunContextWrapper, note_id: str) -> str:
"""Delete a note. For wiki notes, also removes the rendered Markdown file. """Delete a note.
Args: Args:
note_id: Note id to delete. note_id: Note id to delete.