Compare commits

...

15 Commits

19 changed files with 649 additions and 615 deletions
+1 -1
View File
@@ -1,3 +1,3 @@
{
"recommendations": ["dbaeumer.vscode-eslint", "prettier.prettier-vscode"]
"recommendations": ["dbaeumer.vscode-eslint", "esbenp.prettier-vscode"]
}
+2 -2
View File
@@ -69,10 +69,10 @@ members of the project's leadership.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][https://www.contributor-covenant.org], version 1.4,
This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org), version 1.4,
available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
----
---
> Chinese Version
> 《阿里巴巴开源行为准则》
+14 -16
View File
@@ -20,34 +20,38 @@
## ✨ Features
- **🎯 轻松集成**
- 无需 Python,无需无头浏览器,无需浏览器插件。纯页面内脚本。
- **🔐 端侧运行**
- **🧠 HTML 脱水**
- **💬 自然语言接口**
- **🎨 HITL 交互界面**
## 🗺️ Roadmap
以及 😉
👉 [**Roadmap**](https://github.com/alibaba/page-agent/issues/96)
- **🧪 实验性的 Chrome 扩展,支持跨页面控制** - `-b feat/ext`
👉 [**🗺️ Roadmap**](https://github.com/alibaba/page-agent/issues/96)
## 🚀 快速开始
### 快速体验 (With Testing LLM)
### 一行代码集成
最快的体验方式
通过我们免费的 Demo LLM 快速体验 PageAgent
```html
<script
src="https://registry.npmmirror.com/page-agent/1.0.0-beta.2/files/dist/iife/page-agent.demo.js"
src="https://registry.npmmirror.com/page-agent/1.0.0/files/dist/iife/page-agent.demo.js"
crossorigin="true"
></script>
```
> ⚠️ **仅用于技术评估。** Demo 模型有速率限制和使用限制,生产环境请使用 NPM 方式
> - **⚠️ 仅用于技术评估。** Demo LLM 有速率和使用限制,可能随时变更
> - **🌷 建议使用自己的 LLM API。**
| 镜像 | URL |
| ------ | ----------------------------------------------------------------------------------------- |
| Global | https://cdn.jsdelivr.net/npm/page-agent@1.0.0-beta.2/dist/iife/page-agent.demo.js |
| China | https://registry.npmmirror.com/page-agent/1.0.0-beta.2/files/dist/iife/page-agent.demo.js |
| Global | https://cdn.jsdelivr.net/npm/page-agent@1.0.0/dist/iife/page-agent.demo.js |
| China | https://registry.npmmirror.com/page-agent/1.0.0/files/dist/iife/page-agent.demo.js |
### NPM 安装
@@ -86,15 +90,9 @@ packages/
## 🤝 贡献
欢迎社区贡献!以下是参与方式:
欢迎社区贡献!请参阅 [CONTRIBUTING.md](CONTRIBUTING.md) 了解环境配置和本地开发说明。
1. Fork 并 clone。`git clone https://github.com/alibaba/page-agent.git && cd page-agent`
2. 安装依赖: `npm install`
3. 启动开发: `npm start`
更多详情请参阅 [CONTRIBUTING.md](CONTRIBUTING.md)。
请在贡献前阅读我们的[行为准则](CODE_OF_CONDUCT.md)。
请在贡献前阅读[行为准则](CODE_OF_CONDUCT.md)。
## 👏 致谢
+15 -17
View File
@@ -19,35 +19,39 @@ The GUI Agent Living in Your Webpage. Control web interfaces with natural langua
## ✨ Features
- **🎯 Easy Integration** - Transform your webpage into an agent with a single script tag.
- **🎯 Easy Integration**
- No python. No headless browser. No browser extension. Just in-page scripts.
- **🔐 Client-Side Processing**
- **🧠 DOM Extraction**
- **💬 Natural Language Interface**
- **🎨 UI with Human in the loop**
## 🗺️ Roadmap
And 😉
👉 [**Roadmap**](https://github.com/alibaba/page-agent/issues/96)
- **🧪 `cross-page` control with an experimental chrome extension** - `-b feat/ext`
👉 [**🗺️ Roadmap**](https://github.com/alibaba/page-agent/issues/96)
## 🚀 Quick Start
### Quick Try (With Testing LLM)
### One-line integration
Fastest way to try PageAgent:
Fastest way to try PageAgent with our free Demo LLM:
```html
<script
src="https://cdn.jsdelivr.net/npm/page-agent@1.0.0-beta.2/dist/iife/page-agent.demo.js"
src="https://cdn.jsdelivr.net/npm/page-agent@1.0.0/dist/iife/page-agent.demo.js"
crossorigin="true"
></script>
```
> ⚠️ **For technical evaluation only.** Demo model has rate limits and usage restrictions. Use NPM for production.
> - **⚠️ For technical evaluation only.** Demo LLM has rate limits and usage restrictions. May change without notice.
> - **🌷 Bring your own LLM API.**
| Mirrors | URL |
| ------- | ----------------------------------------------------------------------------------------- |
| Global | https://cdn.jsdelivr.net/npm/page-agent@1.0.0-beta.2/dist/iife/page-agent.demo.js |
| China | https://registry.npmmirror.com/page-agent/1.0.0-beta.2/files/dist/iife/page-agent.demo.js |
| Global | https://cdn.jsdelivr.net/npm/page-agent@1.0.0/dist/iife/page-agent.demo.js |
| China | https://registry.npmmirror.com/page-agent/1.0.0/files/dist/iife/page-agent.demo.js |
### NPM Installation
@@ -84,15 +88,9 @@ packages/
## 🤝 Contributing
We welcome contributions from the community! Here's how to get started:
We welcome contributions from the community! Follow our instructions in [CONTRIBUTING.md](CONTRIBUTING.md) for environment setup and local development.
1. Fork and clone. `git clone https://github.com/alibaba/page-agent.git && cd page-agent`
2. Install dependencies: `npm install`
3. Start development: `npm start`
More details in [CONTRIBUTING.md](CONTRIBUTING.md).
Please read our [Code of Conduct](CODE_OF_CONDUCT.md) before contributing.
Please read [Code of Conduct](CODE_OF_CONDUCT.md) before contributing.
## 👏 Acknowledgments
+525 -539
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "root",
"private": true,
"version": "1.0.0-beta.2",
"version": "1.0.0",
"type": "module",
"workspaces": [
"packages/page-controller",
+3 -3
View File
@@ -1,7 +1,7 @@
{
"name": "@page-agent/core",
"private": false,
"version": "1.0.0-beta.2",
"version": "1.0.0",
"type": "module",
"main": "./dist/esm/page-agent-core.js",
"module": "./dist/esm/page-agent-core.js",
@@ -45,7 +45,7 @@
"dependencies": {
"chalk": "^5.6.2",
"zod": "^4.3.5",
"@page-agent/llms": "1.0.0-beta.2",
"@page-agent/page-controller": "1.0.0-beta.2"
"@page-agent/llms": "1.0.0",
"@page-agent/page-controller": "1.0.0"
}
}
+6 -6
View File
@@ -15,7 +15,7 @@ import {
AgentActivity,
AgentReflection,
AgentStatus,
AgentStep,
AgentStepEvent,
ExecutionResult,
HistoricalEvent,
MacroToolInput,
@@ -26,6 +26,7 @@ import { assert } from './utils/assert'
export { type PageAgentConfig }
export { tool, type PageAgentTool } from './tools'
export type * from './types'
/**
* AI agent for browser DOM automation.
@@ -98,8 +99,7 @@ export class PageAgentCore extends EventTarget {
this.emitActivity({ type: 'retrying', attempt, maxAttempts })
// Also push to history for panel rendering
this.history.push({
type: 'error',
errorType: 'retry',
type: 'retry',
message: `LLM retry attempt ${attempt} of ${maxAttempts}`,
attempt,
maxAttempts,
@@ -113,7 +113,6 @@ export class PageAgentCore extends EventTarget {
// Also push to history for panel rendering
this.history.push({
type: 'error',
errorType: 'error',
message,
rawResponse: (error as InvokeError).rawResponse,
})
@@ -255,7 +254,7 @@ export class PageAgentCore extends EventTarget {
next_goal: input.next_goal,
}
const actionName = Object.keys(input.action)[0]
const action: AgentStep['action'] = {
const action: AgentStepEvent['action'] = {
name: actionName,
input: input.action[actionName],
output: output,
@@ -263,11 +262,12 @@ export class PageAgentCore extends EventTarget {
this.history.push({
type: 'step',
stepIndex: step,
reflection,
action,
usage: result.usage,
rawResponse: result.rawResponse,
} as AgentStep)
} as AgentStepEvent)
this.#emitHistoryChange()
console.log(chalk.green('Step finished:'), actionName)
+21 -8
View File
@@ -33,8 +33,9 @@ export interface MacroToolResult {
/**
* A single agent step with reflection and action
*/
export interface AgentStep {
export interface AgentStepEvent {
type: 'step'
stepIndex: number
reflection: Partial<AgentReflection>
action: {
name: string
@@ -68,21 +69,33 @@ export interface UserTakeoverEvent {
}
/**
* Error event (retry or error from LLM)
* Retry event - LLM call is being retried
*/
export interface ErrorEvent {
type: 'error'
errorType: 'retry' | 'error'
export interface RetryEvent {
type: 'retry'
message: string
attempt: number
maxAttempts: number
}
/**
* Error event - fatal error from LLM or execution
*/
export interface AgentErrorEvent {
type: 'error'
message: string
attempt?: number
maxAttempts?: number
rawResponse?: unknown
}
/**
* Union type for all history events
*/
export type HistoricalEvent = AgentStep | ObservationEvent | UserTakeoverEvent | ErrorEvent
export type HistoricalEvent =
| AgentStepEvent
| ObservationEvent
| UserTakeoverEvent
| RetryEvent
| AgentErrorEvent
/**
* Agent execution status
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@page-agent/llms",
"version": "1.0.0-beta.2",
"version": "1.0.0",
"type": "module",
"main": "./dist/lib/page-agent-llms.js",
"module": "./dist/lib/page-agent-llms.js",
+5 -5
View File
@@ -1,7 +1,7 @@
{
"name": "page-agent",
"private": false,
"version": "1.0.0-beta.2",
"version": "1.0.0",
"type": "module",
"main": "./dist/esm/page-agent.js",
"module": "./dist/esm/page-agent.js",
@@ -46,9 +46,9 @@
"dependencies": {
"chalk": "^5.6.2",
"zod": "^4.3.5",
"@page-agent/llms": "1.0.0-beta.2",
"@page-agent/page-controller": "1.0.0-beta.2",
"@page-agent/core": "1.0.0-beta.2",
"@page-agent/ui": "1.0.0-beta.2"
"@page-agent/llms": "1.0.0",
"@page-agent/page-controller": "1.0.0",
"@page-agent/core": "1.0.0",
"@page-agent/ui": "1.0.0"
}
}
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@page-agent/page-controller",
"version": "1.0.0-beta.2",
"version": "1.0.0",
"type": "module",
"main": "./dist/lib/page-controller.js",
"module": "./dist/lib/page-controller.js",
@@ -80,6 +80,9 @@ export class PageController extends EventTarget {
/** last time the tree was updated */
private lastTimeUpdate = 0
/** Whether the tree has been indexed at least once */
private isIndexed = false
/** Visual mask overlay for blocking user interaction during automation */
private mask: InstanceType<typeof import('./mask/SimulatorMask').SimulatorMask> | null = null
private maskReady: Promise<void> | null = null
@@ -196,6 +199,9 @@ export class PageController extends EventTarget {
this.elementTextMap.clear()
this.elementTextMap = dom.getElementTextMap(this.simplifiedHTML)
// Mark as indexed - now element actions are allowed
this.isIndexed = true
// Restore mask blocking
if (this.mask) {
this.mask.wrapper.style.pointerEvents = 'auto'
@@ -215,11 +221,22 @@ export class PageController extends EventTarget {
// ======= Element Actions =======
/**
* Ensure the tree has been indexed before any index-based operation.
* Throws if updateTree() hasn't been called yet.
*/
private assertIndexed(): void {
if (!this.isIndexed) {
throw new Error('DOM tree not indexed. Can not perform actions on elements.')
}
}
/**
* Click element by index
*/
async clickElement(index: number): Promise<ActionResult> {
try {
this.assertIndexed()
const element = getElementByIndex(this.selectorMap, index)
const elemText = this.elementTextMap.get(index)
await clickElement(element)
@@ -249,6 +266,7 @@ export class PageController extends EventTarget {
*/
async inputText(index: number, text: string): Promise<ActionResult> {
try {
this.assertIndexed()
const element = getElementByIndex(this.selectorMap, index)
const elemText = this.elementTextMap.get(index)
await inputTextElement(element, text)
@@ -270,6 +288,7 @@ export class PageController extends EventTarget {
*/
async selectOption(index: number, optionText: string): Promise<ActionResult> {
try {
this.assertIndexed()
const element = getElementByIndex(this.selectorMap, index)
const elemText = this.elementTextMap.get(index)
await selectOptionElement(element as HTMLSelectElement, optionText)
@@ -298,6 +317,8 @@ export class PageController extends EventTarget {
try {
const { down, numPages, pixels, index } = options
this.assertIndexed()
const scrollAmount = pixels ?? numPages * (down ? 1 : -1) * window.innerHeight
const element = index !== undefined ? getElementByIndex(this.selectorMap, index) : null
@@ -327,6 +348,8 @@ export class PageController extends EventTarget {
try {
const { right, pixels, index } = options
this.assertIndexed()
const scrollAmount = pixels * (right ? 1 : -1)
const element = index !== undefined ? getElementByIndex(this.selectorMap, index) : null
@@ -394,6 +417,7 @@ export class PageController extends EventTarget {
this.selectorMap.clear()
this.elementTextMap.clear()
this.simplifiedHTML = '<EMPTY>'
this.isIndexed = false
this.mask?.dispose()
this.mask = null
}
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@page-agent/ui",
"version": "1.0.0-beta.2",
"version": "1.0.0",
"type": "module",
"main": "./dist/lib/page-agent-ui.js",
"module": "./dist/lib/page-agent-ui.js",
+18 -9
View File
@@ -563,9 +563,8 @@ export class Panel {
// 2. Render each history event
const history = this.#agent.history
for (let i = 0; i < history.length; i++) {
const event = history[i]
items.push(...this.#createHistoryCards(event, i + 1))
for (const event of history) {
items.push(...this.#createHistoryCards(event))
}
this.#historySection.innerHTML = items.join('')
@@ -577,14 +576,17 @@ export class Panel {
}
/** Create cards for a history event */
#createHistoryCards(event: PanelAgentAdapter['history'][number], stepNumber: number): string[] {
#createHistoryCards(event: PanelAgentAdapter['history'][number]): string[] {
const cards: string[] = []
const time = formatTime(this.#config.language ?? 'en-US')
const meta = this.#i18n.t('ui.panel.step', {
number: stepNumber.toString(),
time,
duration: '',
})
const meta =
event.type === 'step' && event.stepIndex !== undefined
? this.#i18n.t('ui.panel.step', {
number: (event.stepIndex + 1).toString(),
time,
duration: '',
})
: time
if (event.type === 'step') {
// Reflection card
@@ -606,6 +608,13 @@ export class Panel {
)
} else if (event.type === 'user_takeover') {
cards.push(createCard({ icon: '👤', content: 'User takeover', meta, type: 'input' }))
} else if (event.type === 'retry') {
const retryInfo = `${event.message || 'Retrying'} (${event.attempt}/${event.maxAttempts})`
cards.push(createCard({ icon: '🔄', content: retryInfo, meta, type: 'observation' }))
} else if (event.type === 'error') {
cards.push(
createCard({ icon: '❌', content: event.message || 'Error', meta, type: 'observation' })
)
}
return cards
+7 -1
View File
@@ -33,7 +33,8 @@ export interface PanelAgentAdapter extends EventTarget {
/** History of agent events */
readonly history: readonly {
type: 'step' | 'observation' | 'user_takeover' | 'error'
type: 'step' | 'observation' | 'user_takeover' | 'retry' | 'error'
stepIndex?: number
/** For 'step' type */
reflection?: {
evaluation_previous_goal?: string
@@ -48,6 +49,11 @@ export interface PanelAgentAdapter extends EventTarget {
}
/** For 'observation' type */
content?: string
/** For 'retry' type */
attempt?: number
maxAttempts?: number
/** For 'retry' and 'error' types */
message?: string
}[]
/** Current task being executed */
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@page-agent/website",
"private": true,
"version": "1.0.0-beta.2",
"version": "1.0.0",
"type": "module",
"scripts": {
"dev": "vite --host 0.0.0.0",
+2 -2
View File
@@ -1,8 +1,8 @@
// Demo build (auto-init with demo LLM, for quick testing)
export const CDN_DEMO_URL =
'https://cdn.jsdelivr.net/npm/page-agent@1.0.0-beta.2/dist/iife/page-agent.demo.js'
'https://cdn.jsdelivr.net/npm/page-agent@1.0.0/dist/iife/page-agent.demo.js'
export const CDN_DEMO_CN_URL =
'https://registry.npmmirror.com/page-agent/1.0.0-beta.2/files/dist/iife/page-agent.demo.js'
'https://registry.npmmirror.com/page-agent/1.0.0/files/dist/iife/page-agent.demo.js'
// Demo LLM for website testing
export const DEMO_MODEL = 'PAGE-AGENT-FREE-TESTING-RANDOM'
+1 -1
View File
@@ -79,7 +79,7 @@ for (const pkg of packages) {
let pkgChanged = false
// Update package version
if (oldVersion !== newVersion) {
if (pkgJson.version !== newVersion) {
pkgJson.version = newVersion
pkgChanged = true
}