Compare commits

..

6 Commits

Author SHA1 Message Date
Simon 1751ba325d chore(version): bump version to 0.0.22 2026-01-13 21:01:16 +08:00
Simon 066b045822 Merge pull request #108 from alibaba/feat/dual-umd
feat: add `cdn` packages for iife builds
2026-01-13 20:58:37 +08:00
Simon 3b21c42cdb feat: add cdn packages for iife builds 2026-01-13 20:52:41 +08:00
Simon 8b1e91d665 Merge pull request #107 from alibaba/refactor/mv-simulator-mask-to-page-controller
Refactor: move simulator mask to page controller
2026-01-13 17:38:37 +08:00
Simon aa5909264a chore: mv mask code to folder 2026-01-13 17:35:24 +08:00
Simon b6232d4e21 refactor: mv SimulatorMask from ui to page-controller 2026-01-13 17:33:38 +08:00
25 changed files with 281 additions and 69 deletions
+29 -25
View File
@@ -9,9 +9,10 @@ This is a **monorepo** with npm workspaces:
Internal packages:
- **CDN** (`packages/cdn/`) - IIFE builds for script tag usage (npm: `@page-agent/cdn`)
- **LLMs** (`packages/llms/`) - LLM client with reflection-before-action mental model
- **Page Controller** (`packages/page-controller/`) - DOM operations, independent of LLM
- **UI** (`packages/ui/`) - Panel, SimulatorMask, i18n. Decoupled from PageAgent
- **Page Controller** (`packages/page-controller/`) - DOM operations and visual feedback (SimulatorMask), independent of LLM
- **UI** (`packages/ui/`) - Panel and i18n. Decoupled from PageAgent
## Development Commands
@@ -31,6 +32,7 @@ Simple monorepo solution: TypeScript references + Vite aliases. Update tsconfig
```
packages/
├── page-agent/ # npm: "page-agent" ⭐ MAIN
├── cdn/ # npm: "@page-agent/cdn" (IIFE builds)
├── website/ # @page-agent/website (private)
├── llms/ # @page-agent/llms
├── page-controller/ # @page-agent/page-controller
@@ -43,8 +45,8 @@ packages/
- **Page Agent**: Core lib. Imports from `@page-agent/llms`, `@page-agent/page-controller`, `@page-agent/ui`
- **LLMs**: LLM client with MacroToolInput contract. No dependency on page-agent
- **UI**: Panel, Mask, i18n. No dependency on page-agent
- **Page Controller**: Pure DOM operations. No LLM or UI dependency
- **UI**: Panel and i18n. No dependency on page-agent
- **Page Controller**: DOM operations with optional visual feedback (SimulatorMask). No LLM dependency. Enable mask via `enableMask: true` config
### PageController ↔ PageAgent Communication
@@ -69,41 +71,43 @@ const pageInfo = await this.pageController.getPageInfo()
3. **LLM Processing**: AI returns action plans (page-agent)
4. **Indexed Operations**: PageAgent calls PageController by element index
### CDN Auto-Injection
### CDN Builds (`packages/cdn/`)
Library auto-initializes via script tag:
Two IIFE builds for script tag usage:
```html
<script src="page-agent.js?model=gpt-4"></script>
```
| Build | File | Description |
| ----- | -------------------- | -------------------------------- |
| Demo | `page-agent.demo.js` | Auto-init with built-in test API |
| Full | `page-agent.js` | Exposes `PageAgent` class only |
Query params configure `PageAgentConfig` in `src/umd.ts`.
Demo build supports query params (e.g., `?model=gpt-4&lang=en-US`).
## Key Files Reference
### Page Agent (`packages/page-agent/`)
| File | Description |
|------|-------------|
| `src/PageAgent.ts` | ⭐ Main AI agent class |
| `src/umd.ts` | CDN/UMD entry with auto-init |
| `src/tools/` | Tool definitions calling PageController |
| File | Description |
| ------------------ | --------------------------------------- |
| `src/PageAgent.ts` | ⭐ Main AI agent class |
| `src/umd.ts` | CDN/UMD entry with auto-init |
| `src/tools/` | Tool definitions calling PageController |
### LLMs (`packages/llms/`)
| File | Description |
|------|-------------|
| `src/index.ts` | ⭐ LLM class with retry logic |
| `src/types.ts` | MacroToolInput, AgentBrain, LLMConfig |
| `src/OpenAILenientClient.ts` | OpenAI-compatible client |
| File | Description |
| ---------------------------- | ------------------------------------- |
| `src/index.ts` | ⭐ LLM class with retry logic |
| `src/types.ts` | MacroToolInput, AgentBrain, LLMConfig |
| `src/OpenAILenientClient.ts` | OpenAI-compatible client |
### Page Controller (`packages/page-controller/`)
| File | Description |
|------|-------------|
| `src/PageController.ts` | ⭐ Main controller class |
| `src/actions.ts` | Element interactions (click, input, scroll) |
| `src/dom/dom_tree/index.js` | Core DOM extraction engine |
| File | Description |
| --------------------------- | ---------------------------------------------------------- |
| `src/PageController.ts` | ⭐ Main controller class with optional mask support |
| `src/SimulatorMask.ts` | Visual overlay blocking user interaction during automation |
| `src/actions.ts` | Element interactions (click, input, scroll) |
| `src/dom/dom_tree/index.js` | Core DOM extraction engine |
## Adding New Features
+3 -2
View File
@@ -79,8 +79,9 @@ PageAgent adopts a simplified monorepo structure:
packages/
├── page-agent/ # AI agent (npm: page-agent)
├── llms/ # LLM 客户端 (npm: @page-agent/llms)
├── page-controller/ # DOM 操作 (npm: @page-agent/page-controller)
├── ui/ # 面板 & 蒙层 & 模拟鼠标 (npm: @page-agent/ui)
├── page-controller/ # DOM 操作 & 蒙层 & 模拟鼠标 (npm: @page-agent/page-controller)
├── ui/ # 面板 & i18n (npm: @page-agent/ui)
├── cdn/ # CDN IIFE builds (npm: @page-agent/cdn)
└── website/ # 文档站点
```
+3 -2
View File
@@ -79,8 +79,9 @@ PageAgent adopts a simplified monorepo structure:
packages/
├── page-agent/ # AI agent (npm: page-agent)
├── llms/ # LLM client (npm: @page-agent/llms)
├── page-controller/ # DOM operations (npm: @page-agent/page-controller)
├── ui/ # Panel & Mask & Mouse Animation (npm: @page-agent/ui)
├── page-controller/ # DOM operations & Visual Mask (npm: @page-agent/page-controller)
├── ui/ # Panel & i18n (npm: @page-agent/ui)
├── cdn/ # CDN IIFE builds (npm: @page-agent/cdn)
└── website/ # Demo & Documentation site
```
+27 -14
View File
@@ -1,18 +1,19 @@
{
"name": "root",
"version": "0.0.21",
"version": "0.0.22",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "root",
"version": "0.0.21",
"version": "0.0.22",
"license": "MIT",
"workspaces": [
"packages/page-controller",
"packages/ui",
"packages/llms",
"packages/page-agent",
"packages/cdn",
"packages/website"
],
"devDependencies": {
@@ -1597,6 +1598,10 @@
"url": "https://github.com/sponsors/epoberezkin"
}
},
"node_modules/@page-agent/cdn": {
"resolved": "packages/cdn",
"link": true
},
"node_modules/@page-agent/llms": {
"resolved": "packages/llms",
"link": true
@@ -8140,9 +8145,17 @@
"zod": "^3.25.0 || ^4.0.0"
}
},
"packages/cdn": {
"name": "@page-agent/cdn",
"version": "0.0.22",
"license": "MIT",
"dependencies": {
"page-agent": "0.0.22"
}
},
"packages/llms": {
"name": "@page-agent/llms",
"version": "0.0.21",
"version": "0.0.22",
"license": "MIT",
"dependencies": {
"chalk": "^5.6.2",
@@ -8150,32 +8163,32 @@
}
},
"packages/page-agent": {
"version": "0.0.21",
"version": "0.0.22",
"license": "MIT",
"dependencies": {
"@page-agent/llms": "0.0.21",
"@page-agent/page-controller": "0.0.21",
"@page-agent/ui": "0.0.21",
"@page-agent/llms": "0.0.22",
"@page-agent/page-controller": "0.0.22",
"@page-agent/ui": "0.0.22",
"chalk": "^5.6.2",
"zod": "^4.3.5"
}
},
"packages/page-controller": {
"name": "@page-agent/page-controller",
"version": "0.0.21",
"license": "MIT"
},
"packages/ui": {
"name": "@page-agent/ui",
"version": "0.0.21",
"version": "0.0.22",
"license": "MIT",
"dependencies": {
"ai-motion": "^0.4.8"
}
},
"packages/ui": {
"name": "@page-agent/ui",
"version": "0.0.22",
"license": "MIT"
},
"packages/website": {
"name": "@page-agent/website",
"version": "0.0.21",
"version": "0.0.22",
"dependencies": {
"@radix-ui/react-icons": "^1.3.2",
"@radix-ui/react-separator": "^1.1.8",
+2 -1
View File
@@ -1,13 +1,14 @@
{
"name": "root",
"private": true,
"version": "0.0.21",
"version": "0.0.22",
"type": "module",
"workspaces": [
"packages/page-controller",
"packages/ui",
"packages/llms",
"packages/page-agent",
"packages/cdn",
"packages/website"
],
"description": "AI-powered UI agent for web applications",
+23
View File
@@ -0,0 +1,23 @@
{
"name": "@page-agent/cdn",
"private": false,
"version": "0.0.22",
"type": "module",
"files": [
"dist/"
],
"description": "CDN builds for page-agent - IIFE bundles for script tag usage",
"author": "Simon<gaomeng1900>",
"license": "MIT",
"repository": {
"type": "git",
"url": "https://github.com/alibaba/page-agent.git"
},
"homepage": "https://alibaba.github.io/page-agent/",
"scripts": {
"build": "vite build && vite build --mode demo"
},
"dependencies": {
"page-agent": "0.0.22"
}
}
+59
View File
@@ -0,0 +1,59 @@
// @ts-nocheck
/**
* Demo CDN build for page-agent
* Auto-initializes with built-in demo API for testing
*
* Usage:
* <script src="https://unpkg.com/@page-agent/cdn/dist/page-agent.demo.js"></script>
*/
import { PageAgent } from 'page-agent'
// Clean up existing instances to prevent multiple injections from bookmarklet
if (window.pageAgent) {
window.pageAgent.dispose()
}
// Mount to global window object
window.PageAgent = PageAgent
// Export for IIFE
export { PageAgent }
console.log('🚀 page-agent.js loaded!')
const DEMO_MODEL = 'PAGE-AGENT-FREE-TESTING-RANDOM'
const DEMO_BASE_URL = 'https://hwcxiuzfylggtcktqgij.supabase.co/functions/v1/llm-testing-proxy'
const DEMO_API_KEY = 'PAGE-AGENT-FREE-TESTING-RANDOM'
// Demo warning
console.log(
'%c⚠️ DEMO MODE %c\n' +
'This build uses a shared testing LLM with rate limits.\n' +
'For production, use page-agent.js with your own API key:\n' +
'https://alibaba.github.io/page-agent/#/docs/features/models',
'background: #f59e0b; color: #000; font-size: 14px; font-weight: bold; padding: 4px 8px; border-radius: 4px;',
'color: #f59e0b; font-size: 12px;'
)
// in case document.x is not ready yet
setTimeout(() => {
const currentScript = document.currentScript
if (currentScript) {
console.log('🚀 page-agent.js detected current script:', currentScript.src)
const url = new URL(currentScript.src)
const model = url.searchParams.get('model') || DEMO_MODEL
const baseURL = url.searchParams.get('baseURL') || DEMO_BASE_URL
const apiKey = url.searchParams.get('apiKey') || DEMO_API_KEY
const language = url.searchParams.get('lang') || 'zh-CN'
const config = { model, baseURL, apiKey, language }
window.pageAgent = new PageAgent(config)
} else {
console.log('🚀 page-agent.js no current script detected, using default demo config')
window.pageAgent = new PageAgent()
}
console.log('🚀 page-agent.js initialized with config:', window.pageAgent.config)
window.pageAgent.panel.show()
})
+15
View File
@@ -0,0 +1,15 @@
// @ts-nocheck
/**
* Full CDN build for page-agent
* Exposes PageAgent class for manual instantiation and configuration
*
* Usage:
* <script src="https://unpkg.com/@page-agent/cdn/dist/page-agent.js"></script>
* <script>
* const agent = new PageAgent({ model: 'gpt-4o', apiKey: 'your-key' })
* agent.panel.show()
* </script>
*/
import { PageAgent } from 'page-agent'
export { PageAgent }
+42
View File
@@ -0,0 +1,42 @@
// @ts-nocheck
import { dirname, resolve } from 'path'
import { fileURLToPath } from 'url'
import { defineConfig } from 'vite'
import cssInjectedByJsPlugin from 'vite-plugin-css-injected-by-js'
const __dirname = dirname(fileURLToPath(import.meta.url))
/**
* CDN IIFE builds for page-agent
*
* Build targets (via --mode):
* - demo: page-agent.demo.js with auto-init and built-in API
* - full (default): page-agent.js exposes PageAgent class only
*/
export default defineConfig(({ mode }) => {
const isDemo = mode === 'demo'
const entry = isDemo ? resolve(__dirname, 'src/demo.js') : resolve(__dirname, 'src/full.js')
const fileName = isDemo ? 'page-agent.demo' : 'page-agent'
return {
plugins: [cssInjectedByJsPlugin({ relativeCSSInjection: true })],
publicDir: false,
esbuild: {
keepNames: true,
},
build: {
lib: {
entry,
name: 'PageAgent',
fileName: () => `${fileName}.js`,
formats: ['iife'],
},
outDir: resolve(__dirname, 'dist'),
emptyOutDir: !isDemo, // only empty on first build (full)
minify: false,
},
define: {
'process.env.NODE_ENV': '"production"',
},
}
})
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@page-agent/llms",
"version": "0.0.21",
"version": "0.0.22",
"type": "module",
"main": "./dist/lib/page-agent-llms.js",
"module": "./dist/lib/page-agent-llms.js",
+4 -4
View File
@@ -1,7 +1,7 @@
{
"name": "page-agent",
"private": false,
"version": "0.0.21",
"version": "0.0.22",
"type": "module",
"main": "./dist/esm/page-agent.js",
"module": "./dist/esm/page-agent.js",
@@ -46,8 +46,8 @@
"dependencies": {
"chalk": "^5.6.2",
"zod": "^4.3.5",
"@page-agent/llms": "0.0.21",
"@page-agent/page-controller": "0.0.21",
"@page-agent/ui": "0.0.21"
"@page-agent/llms": "0.0.22",
"@page-agent/page-controller": "0.0.22",
"@page-agent/ui": "0.0.22"
}
}
+8 -10
View File
@@ -4,7 +4,7 @@
*/
import { LLM, type Tool } from '@page-agent/llms'
import { PageController } from '@page-agent/page-controller'
import { Panel, SimulatorMask } from '@page-agent/ui'
import { Panel } from '@page-agent/ui'
import chalk from 'chalk'
import zod from 'zod'
@@ -92,8 +92,6 @@ export class PageAgent extends EventTarget {
/** PageController for DOM operations */
pageController: PageController
/** Fullscreen mask */
mask = new SimulatorMask()
/** History records */
history: AgentHistory[] = []
@@ -114,8 +112,11 @@ export class PageAgent extends EventTarget {
})
this.tools = new Map(tools)
// Initialize PageController with config
this.pageController = new PageController(this.config)
// Initialize PageController with config (mask enabled by default)
this.pageController = new PageController({
...this.config,
enableMask: this.config.enableMask ?? true,
})
// Listen to LLM events
this.#llmRetryListener = (e) => {
@@ -162,7 +163,7 @@ export class PageAgent extends EventTarget {
await onBeforeTask.call(this)
// Show mask and panel
this.mask.show()
this.pageController.showMask()
this.panel.show()
this.panel.reset()
@@ -485,7 +486,7 @@ export class PageAgent extends EventTarget {
// Task completed
this.panel.update({ type: 'completed' })
this.mask.hide()
this.pageController.hideMask()
this.#abortController.abort()
}
@@ -496,9 +497,7 @@ export class PageAgent extends EventTarget {
const pi = await this.pageController.getPageInfo()
const viewportExpansion = await this.pageController.getViewportExpansion()
this.mask.wrapper.style.pointerEvents = 'none'
await this.pageController.updateTree()
this.mask.wrapper.style.pointerEvents = 'auto'
let simplifiedHTML = await this.pageController.getSimplifiedHTML()
@@ -545,7 +544,6 @@ export class PageAgent extends EventTarget {
this.disposed = true
this.pageController.dispose()
this.panel.dispose()
this.mask.dispose()
this.history = []
this.#abortController.abort(reason ?? 'PageAgent disposed')
+4 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@page-agent/page-controller",
"version": "0.0.21",
"version": "0.0.22",
"type": "module",
"main": "./dist/lib/page-controller.js",
"module": "./dist/lib/page-controller.js",
@@ -34,5 +34,8 @@
"build": "vite build",
"prepublishOnly": "node -e \"const fs=require('fs');['LICENSE'].forEach(f=>fs.copyFileSync('../../'+f,f))\"",
"postpublish": "node -e \"['LICENSE'].forEach(f=>{try{require('fs').unlinkSync(f)}catch{}})\""
},
"dependencies": {
"ai-motion": "^0.4.8"
}
}
@@ -18,13 +18,18 @@ import { VIEWPORT_EXPANSION } from './constants'
import * as dom from './dom'
import type { FlatDomTree, InteractiveElementDomNode } from './dom/dom_tree/type'
import { getPageInfo } from './dom/getPageInfo'
import { SimulatorMask } from './mask/SimulatorMask'
import { patchReact } from './patches/react'
export { SimulatorMask }
/**
* Configuration for PageController
*/
export interface PageControllerConfig extends dom.DomConfig {
viewportExpansion?: number
/** Enable visual mask overlay during operations (default: false) */
enableMask?: boolean
}
interface ActionResult {
@@ -64,12 +69,19 @@ export class PageController extends EventTarget {
/** last time the tree was updated */
private lastTimeUpdate = 0
/** Visual mask overlay for blocking user interaction during automation */
private mask: SimulatorMask | null = null
constructor(config: PageControllerConfig = {}) {
super()
this.config = config
patchReact(this)
if (config.enableMask) {
this.mask = new SimulatorMask()
}
}
// ======= State Queries =======
@@ -136,12 +148,18 @@ export class PageController extends EventTarget {
/**
* Update DOM tree, returns simplified HTML for LLM.
* This is the main method to refresh the page state.
* Automatically bypasses mask during DOM extraction if enabled.
*/
async updateTree(): Promise<string> {
this.dispatchEvent(new Event('beforeUpdate'))
this.lastTimeUpdate = Date.now()
// Temporarily bypass mask to allow DOM extraction
if (this.mask) {
this.mask.wrapper.style.pointerEvents = 'none'
}
dom.cleanUpHighlights()
const blacklist = [
@@ -162,6 +180,11 @@ export class PageController extends EventTarget {
this.elementTextMap.clear()
this.elementTextMap = dom.getElementTextMap(this.simplifiedHTML)
// Restore mask blocking
if (this.mask) {
this.mask.wrapper.style.pointerEvents = 'auto'
}
this.dispatchEvent(new Event('afterUpdate'))
return this.simplifiedHTML
@@ -326,6 +349,24 @@ export class PageController extends EventTarget {
}
}
// ======= Mask Operations =======
/**
* Show the visual mask overlay.
* Only works if enableMask was set to true in config.
*/
showMask(): void {
this.mask?.show()
}
/**
* Hide the visual mask overlay.
* Only works if enableMask was set to true in config.
*/
hideMask(): void {
this.mask?.hide()
}
/**
* Dispose and clean up resources
*/
@@ -335,5 +376,7 @@ export class PageController extends EventTarget {
this.selectorMap.clear()
this.elementTextMap.clear()
this.simplifiedHTML = '<EMPTY>'
this.mask?.dispose()
this.mask = null
}
}
+6
View File
@@ -0,0 +1,6 @@
/// <reference types="vite/client" />
declare module '*.module.css' {
const classes: Record<string, string>
export default classes
}
+1 -1
View File
@@ -29,7 +29,7 @@ export default defineConfig({
},
outDir: resolve(__dirname, 'dist', 'lib'),
rollupOptions: {
external: ['@page-agent/*'],
external: ['@page-agent/*', 'ai-motion'],
},
minify: false,
sourcemap: true,
+2 -5
View File
@@ -1,6 +1,6 @@
{
"name": "@page-agent/ui",
"version": "0.0.21",
"version": "0.0.22",
"type": "module",
"main": "./dist/lib/page-agent-ui.js",
"module": "./dist/lib/page-agent-ui.js",
@@ -15,7 +15,7 @@
"files": [
"dist/"
],
"description": "UI components for page-agent - Panel, SimulatorMask, and i18n",
"description": "UI components for page-agent - Panel and i18n",
"keywords": [
"page-agent",
"ui",
@@ -34,8 +34,5 @@
"build": "vite build",
"prepublishOnly": "node -e \"const fs=require('fs');['LICENSE'].forEach(f=>fs.copyFileSync('../../'+f,f))\"",
"postpublish": "node -e \"['LICENSE'].forEach(f=>{try{require('fs').unlinkSync(f)}catch{}})\""
},
"dependencies": {
"ai-motion": "^0.4.8"
}
}
-1
View File
@@ -1,4 +1,3 @@
export { Panel, type PanelConfig, type PanelUpdate } from './Panel'
export { SimulatorMask } from './SimulatorMask'
export { UIState, type Step, type AgentStatus } from './UIState'
export { I18n, type SupportedLanguage, type TranslationKey } from './i18n'
+1 -1
View File
@@ -29,7 +29,7 @@ export default defineConfig({
},
outDir: resolve(__dirname, 'dist', 'lib'),
rollupOptions: {
external: ['ai-motion'],
external: [],
},
minify: false,
sourcemap: true,
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@page-agent/website",
"private": true,
"version": "0.0.21",
"version": "0.0.22",
"type": "module",
"scripts": {
"dev": "vite",
@@ -121,6 +121,13 @@ export default function Configuration() {
/** Viewport expansion in pixels (-1 for full page) */
viewportExpansion?: number
/**
* Enable visual mask overlay during automation.
* Blocks user interaction while agent is running.
* Default: false for PageController, true for PageAgent.
*/
enableMask?: boolean
}`}
/>
</section>