Commit Graph

85 Commits

Author SHA1 Message Date
Simon 7dffaa4718 Merge pull request #512 from linked-danis/fix/scroll-predicate-types
Fix scroll predicate types (boolean | null -> boolean)
2026-06-11 21:22:31 +08:00
Simon 56c09a9ae9 test(page-controller): add happy-dom env and basic tests 2026-06-11 21:10:05 +08:00
Simon 9ed0a09194 feat(core): make execute_javascript honor AbortSignal
Expose the task AbortSignal as `signal` in the script scope so cooperative
code can cancel promptly, and re-check signal.throwIfAborted() after the
script settles to discard stale results.

Closes #537.
2026-06-09 21:30:27 +08:00
Simon 01c9cc9ea7 feat(page-controller): refine dark mode detection heuristics 2026-06-09 20:33:14 +08:00
Simon b3ff0a1a25 Merge pull request #373 from tageniu/feat/dark-mode-detect
Implement TODO: enhance dark mode detection
2026-06-09 20:18:33 +08:00
64JohnLee 7b623c5b22 chore(page-controller): remove accidental debug console.log calls
Two debug log statements were left in production code:

- `isScrollableElement()` logged `scrollData!!!` for every scrollable
  element found during DOM tree construction. Because the DOM is rebuilt
  on every agent action, this fired repeatedly and also triggered
  unnecessary JSON serialisation of scroll metrics on the hot path.

- `SimulatorMask.dispose()` logged 'dispose SimulatorMask' every time
  the highlight overlay was torn down.

Neither had a structured prefix or was gated behind a debug flag.
Removing both silences console noise for end users and removes the
serialisation overhead in the scroll-detection hot path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 16:47:04 +08:00
linked-danis 903c2a736c fix: return booleans from scroll predicates 2026-05-18 15:00:52 +02:00
Simon c6f09375f8 fix(controller): isMainContentDark too aggressive 2026-04-20 20:31:50 -04:00
tageniu cc33297cc4 fix(controller): check html and body data attrs independently
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-20 20:31:45 -04:00
Shijie Sun 8556745645 feat(page-controller): enhance dark mode detection
Resolve the @TODO in checkDarkMode.ts by adding 3 new detection
strategies and expanding data-attribute coverage:

- Check data-color-mode, data-bs-theme, data-color-scheme attributes
- Read CSS color-scheme property and <meta name="color-scheme"> tag
- Inspect background of SPA containers (#app, #root, #__next, etc.)
- Detect light text color as a dark-theme signal
2026-04-20 20:14:07 -04:00
Simon a1b7684bf2 Merge pull request #418 from lgy2020/fix/simulator-mask-raf-leak
fix(mask): dispose 后 requestAnimationFrame 循环未停止导致内存泄漏
2026-04-13 20:14:32 +08:00
Simon da67f3b07e chore(controller): .disposed guard 2026-04-13 20:13:41 +08:00
Simon ca197d14e9 fix: dom typing 2026-04-12 03:16:39 +08:00
Simon 4d27d49752 refactor(setup): upgrade to TypeScript 6 with source-first monorepo resolution 2026-04-12 02:04:21 +08:00
liuguiyuan 9104064e8c fix(mask): stop requestAnimationFrame loop on dispose to prevent memory leak
The #moveCursorToTarget() method recursively schedules itself via
requestAnimationFrame, creating a continuous animation loop for the
AI cursor. However, dispose() only removes the DOM wrapper element
without stopping this loop, causing:

- CPU waste: rAF callback continues executing every frame (~60fps)
  after the mask is disposed, performing unnecessary calculations
  on a detached cursor element.
- Resource leak: Each SimulatorMask instance creates an unrecoverable
  animation loop that persists for the lifetime of the page.
- Console noise: style assignments to removed DOM nodes may produce
  browser warnings.

Fix: Add a #disposed boolean flag, checked at the top of
#moveCursorToTarget() to short-circuit the recursion. Set the flag
to true in dispose() before removing DOM elements.

Changes:
- Add #disposed field (default false)
- Guard #moveCursorToTarget() with early return when #disposed
- Set #disposed = true in dispose() before cleanup
2026-04-08 18:53:42 +08:00
Simon 4272939217 fix(controller): treat interactive with aria as distinct 2026-04-03 20:21:11 +08:00
Lubrsy 148bdb6839 fix: recognize role="listitem" as interactive element (#203)
* fix: recognize role="listitem" as interactive element
* fix: DISTINCT_INTERACTIVE_ROLES

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Simon <10131203+gaomeng1900@users.noreply.github.com>
2026-04-03 20:09:48 +08:00
Simon 7e9027167d feat(controller): add keepSemanticTags config to keep the semantic structure of the page 2026-04-03 19:01:54 +08:00
Simon 0ca1c8de0b feat(controller): consider hasScrollbarSignal when detecting scrollable 2026-04-03 16:09:35 +08:00
Simon 85a33ac1a4 feat(controller): improve scroll action 2026-04-02 22:05:47 +08:00
Simon 8159aa58a6 Merge branch 'main' into fix/scroll-direction-pixels 2026-04-02 18:31:56 +08:00
Simon 2bdb3be81c Merge pull request #356 from lgy2020/fix/aria-attribute-detection
fix(isInteractiveCandidate): use hasAttribute with known aria list to detect aria- attributes
2026-04-02 17:41:36 +08:00
Simon bde630f55d chore(controller): add @edit mark 2026-04-02 17:38:02 +08:00
Simon 3efef0ec42 fix(controller): clean up INTERACTIVE_ARIA_ATTRS 2026-04-02 17:33:27 +08:00
Simon 2b20b48dff chore(controller): reuse pointer xy 2026-03-31 20:27:04 +08:00
Simon 296459924a feat(controller): enhance click action with elementFromPoint 2026-03-31 20:02:39 +08:00
Simon 8eee3b27e2 feat(controller): fix SimulatorMast mem leak; add passthrough events 2026-03-31 19:59:57 +08:00
Simon 32d6f0c74b fix(controller): click action robust 2026-03-31 17:41:49 +08:00
liuguiyuan f2b6c9dfd2 fix: use hasAttribute with known aria list for interactive candidate detection
Replace broken el.hasAttribute("aria-") with a curated list of 27
aria attributes checked via hasAttribute. Each check is O(1).

WAI-ARIA 1.2 defines ~50 aria attributes total per MDN.
Of these ~27 appear on interactive elements such as buttons,
inputs, sliders, and dialogs. The remaining ~20 are structural
container attributes like aria-live, aria-colcount, and
aria-rowspan that only appear on non-interactive containers.
Checking them would not change results.
2026-03-31 00:57:44 +08:00
Matt Van Horn 005bc8ab44 fix(page-controller): apply scroll direction to pixels parameter
Two bugs in the scroll direction logic:

1. Vertical scroll with `pixels` ignores the `down` boolean because the
   `??` operator bypasses the direction multiplier when pixels is provided.
   Fix: move the direction multiplier outside the `??` so it applies to
   both the pixels and numPages paths.

2. Horizontal scroll with `pixels` applies direction twice - once in
   PageController.ts and again in actions.ts - causing a double negation
   that reverses the intended direction. Fix: remove the redundant
   direction logic from actions.ts since PageController already signs
   the scroll amount.

Also removes the now-unused `down` and `right` parameters from the
scrollVertically() and scrollHorizontally() action functions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 08:38:32 -07:00
zfangqijun cd2d33a9f6 chore(page-controller): export actions as internal methods 2026-03-21 20:17:01 +08:00
Simon eeb5b6a5af fix(clickElement): reorder iframe scroll and click actions 2026-03-21 02:18:42 +08:00
Simon ad31e2b750 chore: clarify current type handling in isHTMLElement 2026-03-21 02:16:51 +08:00
Simon e98d80b6a0 fix(PageController): same-origin iframe actions 2026-03-21 01:46:09 +08:00
Simon 3459836a14 fix(PageController): lint error 2026-03-20 16:10:43 +08:00
Simon b7b5b6db30 Merge pull request #319 from alibaba/fix/missing-pointerout
fix(PageController): add `mouseleave` event
2026-03-20 16:05:37 +08:00
Simon 05d16313c7 fix(PageController): add mouseleave event 2026-03-20 15:54:36 +08:00
Simon a3a96d85d5 Merge pull request #210 from voidborne-d/fix/contenteditable-fallback
fix: add execCommand fallback for contenteditable input
2026-03-20 15:38:06 +08:00
Simon c89042f142 chore: wording 2026-03-20 15:36:10 +08:00
voidborne-d 2e18bd862d refactor: use const for planASucceeded, clarify LinkedIn comment 2026-03-20 07:08:45 +00:00
Simon 53628532df Merge pull request #251 from linked-danis/pr5-scrollintoview 2026-03-16 18:45:19 +08:00
linked-danis 89546887bd fix: type-safe scrollIntoViewIfNeeded
Add proper interface for WebKit extension method.

Changes:
- Add ScrollableElement interface
- Use typeof check instead of 'as any' cast
2026-03-13 14:22:47 +01:00
linked-danis 6491118cde refactor: SimulatorMask use CSS classes
Replace inline style.display with CSS class toggling.

Changes:
- Add .visible class to CSS module
- Use classList.add/remove instead of style.display

(cherry picked from commit 33465bbf52)
2026-03-13 20:44:34 +08:00
Simon 1628d48c97 chore: align doc styles 2026-03-12 02:20:24 +08:00
d 🔹 2f92a9cb32 fix: add execCommand fallback for contenteditable input (#168)
When typing into contenteditable elements (e.g. LinkedIn post editor),
the synthetic event approach (Plan A) may fail silently — the events
fire but the editor's internal state doesn't update, leaving the
element empty.

This adds an automatic fallback: after Plan A, we verify the text was
actually inserted by checking element.innerText. If it wasn't, we
fall back to execCommand('insertText') which integrates natively with
most rich-text editors including LinkedIn, Quill, and Slate.js.

The fallback uses proper Selection/Range API to select-all before
replacing, and preserves the undo stack since execCommand is handled
by the browser natively.

Fixes #168
2026-03-11 17:07:22 +00:00
caibing de3a6e4660 fix: extract attributes for heuristically-detected interactive elements
Elements detected as interactive via heuristic methods (cursor:pointer
style, interactive class names, event listeners) had empty attributes
because `isInteractiveCandidate()` was used as the gate for attribute
extraction. This function only recognizes standard HTML tags and ARIA
attributes, missing heuristic detections.

After interactivity is confirmed by `isInteractiveElement()`, backfill
attributes for elements that were missed. This ensures
`includeAttributes` (e.g. `['class']`) works correctly for all
interactive elements, not just semantically standard ones.

Closes #124

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 16:38:36 +08:00
Simon 4d931b4430 chore: comments; spell check 2026-03-10 21:02:48 +08:00
JasonOA888 6054ca1217 docs(page-controller): document contenteditable limitations and execCommand fallback
## Changes

Based on @gaomeng1900's comprehensive testing feedback:

1. **Document known limitations**
   - Slate.js and Draft.js do not work with synthetic events
   - Draft.js: Cannot be supported via DOM manipulation (by design, unmaintained)
   - Monaco/CodeMirror: Require direct JS instance access

2. **Clarify tested editors**
   - Works: Quill, LinkedIn, simple contenteditable editors
   - Does not work: Slate.js, Draft.js

3. **Preserve execCommand as fallback**
   - execCommand works better for LinkedIn, Quill, and Draft.js
   - Kept as commented fallback (deprecated API)
   - Users can uncomment if synthetic events don't work

Co-authored-by: gaomeng1900 <gaomeng1900@users.noreply.github.com>
2026-03-10 20:39:20 +08:00
Simon 7f9f0d1589 feat: simplify ContentEditable handling 2026-03-10 19:40:49 +08:00
Simon ddcfa5b499 Merge branch 'main' into fix/contenteditable-input-events 2026-03-10 17:37:20 +08:00