Commit Graph

1 Commits

Author SHA1 Message Date
Akhilesh Arora bf58853343 fix(offload): preserve non-BMP characters in sanitizeText
UNSAFE_CHAR_RE included the surrogate range [\uD800-\uDFFF] without the `u`
flag, so JS treated strings as UTF-16 code units and stripped each half of
every well-formed non-BMP code point. sanitizeText and sanitizeJsonLine
therefore destroyed emoji, CJK Extension B, math bold, etc. in tool params,
tool results, and ref-md archives.

Adding the `u` flag makes paired surrogates combine into a single code point
before matching, so the [\uD800-\uDFFF] entry now matches only lone (malformed)
surrogates, which is the original intent. All other entries (replacement char,
C0/C1 controls, zero-width chars, line separators, BOM) keep their behavior.

Added a vitest suite covering the preserved and stripped cases.

Closes #30
2026-05-16 22:23:45 +02:00