Compare commits
66 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| cb55daf47d | |||
| 43415d3139 | |||
| 9cc9edf17e | |||
| e290e2f67a | |||
| 4272939217 | |||
| 148bdb6839 | |||
| 68cee4cc64 | |||
| 7e9027167d | |||
| 6630774dc9 | |||
| 0ca1c8de0b | |||
| 85a33ac1a4 | |||
| b8fd1aaebc | |||
| 13d952dc9d | |||
| 1f2f5a44d3 | |||
| 8159aa58a6 | |||
| 2bdb3be81c | |||
| bde630f55d | |||
| 3efef0ec42 | |||
| 1141287733 | |||
| ec08b05da1 | |||
| 5ea2edd13c | |||
| 1999bbeda9 | |||
| b3ec571341 | |||
| d0de8ad838 | |||
| 0402a6be03 | |||
| 6823e04ca9 | |||
| 9ae1527201 | |||
| 2b20b48dff | |||
| 296459924a | |||
| 8eee3b27e2 | |||
| 32d6f0c74b | |||
| 6e4094565a | |||
| da2b7b5a8f | |||
| 3207c3ffc1 | |||
| 784627a7c1 | |||
| 46d6c02c95 | |||
| 6d46719fe8 | |||
| f2b6c9dfd2 | |||
| c465fd8aa4 | |||
| 1eef785a61 | |||
| 0b4eb6b49a | |||
| cba54967b8 | |||
| 23dc3b7f7e | |||
| 49b137981c | |||
| cad033d63b | |||
| 312952ec41 | |||
| 52edd78cd4 | |||
| a7c9999d83 | |||
| 7f8ddde748 | |||
| e1fede1194 | |||
| 2322d6c46b | |||
| ac72f55731 | |||
| 5485e852bd | |||
| 48fb65d00b | |||
| 75bc69c0c0 | |||
| 0d9b9e4b9a | |||
| 78a31ef5d1 | |||
| 98005e323a | |||
| f3c253c044 | |||
| 02f63ec04d | |||
| 583f757f0a | |||
| 53c9149d06 | |||
| 64a90e3b3a | |||
| abfa903c9c | |||
| 6dd0ceab73 | |||
| 005bc8ab44 |
@@ -28,7 +28,7 @@ jobs:
|
|||||||
run: npm run build:website
|
run: npm run build:website
|
||||||
|
|
||||||
- name: Setup Pages
|
- name: Setup Pages
|
||||||
uses: actions/configure-pages@v5
|
uses: actions/configure-pages@v6
|
||||||
|
|
||||||
- name: Upload artifact
|
- name: Upload artifact
|
||||||
uses: actions/upload-pages-artifact@v4
|
uses: actions/upload-pages-artifact@v4
|
||||||
@@ -37,4 +37,4 @@ jobs:
|
|||||||
|
|
||||||
- name: Deploy to GitHub Pages
|
- name: Deploy to GitHub Pages
|
||||||
id: deployment
|
id: deployment
|
||||||
uses: actions/deploy-pages@v4
|
uses: actions/deploy-pages@v5
|
||||||
|
|||||||
+13
-171
@@ -2,42 +2,13 @@
|
|||||||
|
|
||||||
♥️ We welcome contributions from everyone.
|
♥️ We welcome contributions from everyone.
|
||||||
|
|
||||||
## 🚀 Quick Start
|
For local development workflows, setup, local LLM config, extension development, testing on other websites, and more details, see [docs/developer-guide.md](docs/developer-guide.md).
|
||||||
|
|
||||||
### Development Setup
|
|
||||||
|
|
||||||
1. **Prerequisites**
|
|
||||||
- `macOS` / `Linux` / `WSL`
|
|
||||||
- `node.js >= 20` with `npm >= 10`
|
|
||||||
- An editor that supports `ts/eslint/prettier`
|
|
||||||
- Make sure `eslint`, `prettier` and `commitlint` work well. Un-linted code won't pass the CI.
|
|
||||||
|
|
||||||
2. **Setup**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
npm i
|
|
||||||
npm start # Start demo and documentation site
|
|
||||||
npm run build # Build libs and website
|
|
||||||
```
|
|
||||||
|
|
||||||
### Project Structure
|
|
||||||
|
|
||||||
This is a **monorepo** with npm workspaces containing **4 main packages**:
|
|
||||||
|
|
||||||
- **Page Agent** (`packages/page-agent/`) - Main entry with built-in UI Panel, published as `page-agent` on npm
|
|
||||||
- **Core** (`packages/core/`) - Core agent logic without UI (npm: `@page-agent/core`)
|
|
||||||
- **Extension** (`packages/extension/`) - Chrome extension for multi-page tasks and browser-level automation
|
|
||||||
- **Website** (`packages/website/`) - React documentation and landing page. Also as demo and test page for the core lib. private package `@page-agent/website`
|
|
||||||
|
|
||||||
> We use a simplified monorepo solution with `native npm-workspace + ts reference + vite alias`. No fancy tooling. Hoisting is required.
|
|
||||||
>
|
|
||||||
> - When developing. Use alias so that we don't have to pre-build.
|
|
||||||
> - When bundling. Use external and disable ts `paths` alias.
|
|
||||||
> - When bundling `IIFE` and `Website`. Bundle everything together.
|
|
||||||
|
|
||||||
## 🤝 How to Contribute
|
## 🤝 How to Contribute
|
||||||
|
|
||||||
### Reporting Issues
|
> **[Maintainer's Note](https://github.com/alibaba/page-agent/issues/349)**
|
||||||
|
|
||||||
|
### Opening Issues
|
||||||
|
|
||||||
- Use the GitHub issue tracker to report bugs or request features
|
- Use the GitHub issue tracker to report bugs or request features
|
||||||
- Search existing issues before creating new ones
|
- Search existing issues before creating new ones
|
||||||
@@ -46,147 +17,24 @@ This is a **monorepo** with npm workspaces containing **4 main packages**:
|
|||||||
|
|
||||||
### Code Contributions
|
### Code Contributions
|
||||||
|
|
||||||
1. **Fork and Clone**
|
1. Follow existing code style and patterns
|
||||||
|
2. Update documentation as needed
|
||||||
```bash
|
3. Add JSDoc for public APIs
|
||||||
git clone https://github.com/your-username/page-agent.git
|
4. Build and lint everything
|
||||||
cd page-agent
|
5. Test in our demo website, and on other websites if applicable
|
||||||
```
|
6. Include screenshots for UI changes
|
||||||
|
|
||||||
2. **Create Feature Branch**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git checkout -b feat/your-feature-name
|
|
||||||
```
|
|
||||||
|
|
||||||
3. **Make Changes**
|
|
||||||
- Follow existing code style and patterns
|
|
||||||
- Add tests for new functionality
|
|
||||||
- Update documentation as needed
|
|
||||||
|
|
||||||
4. **Test Your Changes**
|
|
||||||
- Build and lint everything.
|
|
||||||
- Test in our demo website
|
|
||||||
- Test it on other websites if applicable
|
|
||||||
- `@TODO: test suite`
|
|
||||||
|
|
||||||
5. **Commit and Push**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git add .
|
|
||||||
git commit -m "feat: add awesome feature"
|
|
||||||
git push origin feat/your-feature-name
|
|
||||||
```
|
|
||||||
|
|
||||||
6. **Create Pull Request**
|
|
||||||
- Provide clear description of changes
|
|
||||||
- Link related issues
|
|
||||||
- Include screenshots for UI changes
|
|
||||||
|
|
||||||
## 📝 Code Style
|
|
||||||
|
|
||||||
### General Guidelines
|
|
||||||
|
|
||||||
- Use TypeScript for type safety
|
|
||||||
- Follow existing naming conventions
|
|
||||||
- Write meaningful commit messages
|
|
||||||
- Keep functions small and focused
|
|
||||||
- Add JSDoc for public APIs
|
|
||||||
|
|
||||||
### Vibe Coding with AI
|
### Vibe Coding with AI
|
||||||
|
|
||||||
> [Vibe coding](https://en.wikipedia.org/wiki/Vibe_coding)
|
- Vibe coding is **NOT** allowed for the core lib or the extension!!!
|
||||||
|
|
||||||
- Vibe coding is **RECOMMENDED** when maintaining **the demo, the website, the UI and tests**.
|
- Vibe coding is **RECOMMENDED** when maintaining **the demo, the website, the UI and tests**.
|
||||||
- We have a [website/AGENTS.md](packages/website/AGENTS.md) for that.
|
- Make sure your AI references `AGENTS.md` and `website/AGENTS.md` for better quality.
|
||||||
- Vibe coding is **NOT** allowed for the core lib!!!
|
|
||||||
- NEVER try to vibe coding the MV3 extension!!! It is HELL.
|
|
||||||
- Review anything AI wrote before make a commit. You are the author of anything you commit. NOT AI.
|
- Review anything AI wrote before make a commit. You are the author of anything you commit. NOT AI.
|
||||||
|
|
||||||
If your AI assistant does not support [AGENTS.md](https://agents.md/). Add an alias for it:
|
|
||||||
|
|
||||||
- claude-code (`CLAUDE.md`)
|
|
||||||
|
|
||||||
```markdown
|
|
||||||
@AGENTS.md
|
|
||||||
```
|
|
||||||
|
|
||||||
- antigravity (`.agent/rules/alias.md`)
|
|
||||||
|
|
||||||
```markdown
|
|
||||||
---
|
|
||||||
trigger: always_on
|
|
||||||
---
|
|
||||||
|
|
||||||
@../../AGENTS.md
|
|
||||||
```
|
|
||||||
|
|
||||||
## 🔧 Development Workflows
|
|
||||||
|
|
||||||
### Test With Your Own LLM API
|
|
||||||
|
|
||||||
- Create a `.env` file in the repo root with your LLM API config
|
|
||||||
|
|
||||||
```env
|
|
||||||
LLM_MODEL_NAME=gpt-5.2
|
|
||||||
LLM_API_KEY=your-api-key
|
|
||||||
LLM_BASE_URL=https://api.your-llm-provider.com/v1
|
|
||||||
```
|
|
||||||
|
|
||||||
- **Ollama example** (tested on 0.15 + qwen3:14b, RTX3090 24GB):
|
|
||||||
|
|
||||||
```env
|
|
||||||
LLM_BASE_URL="http://localhost:11434/v1"
|
|
||||||
LLM_API_KEY="NA"
|
|
||||||
LLM_MODEL_NAME="qwen3:14b"
|
|
||||||
```
|
|
||||||
|
|
||||||
> @see https://alibaba.github.io/page-agent/docs/features/models#ollama for configuration
|
|
||||||
|
|
||||||
- **Restart the dev server** to load new env vars
|
|
||||||
- If not provided, the demo will use the free testing proxy by default. By using it, you agree to its [terms](https://github.com/alibaba/page-agent/blob/main/docs/terms-and-privacy.md).
|
|
||||||
|
|
||||||
### Extension Development
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# make sure you ran `npm run build:libs` first
|
|
||||||
# and every time you changed the core libs
|
|
||||||
npm run dev -w @page-agent/ext
|
|
||||||
npm run zip -w @page-agent/ext
|
|
||||||
```
|
|
||||||
|
|
||||||
- Update `packages/extension/docs/extension_api.md` for API integration details
|
|
||||||
|
|
||||||
### Testing on Other Websites
|
|
||||||
|
|
||||||
- Start and serve a local `iife` script
|
|
||||||
|
|
||||||
```bash
|
|
||||||
npm run dev:demo # Serving IIFE with auto rebuild at http://localhost:5174/page-agent.demo.js
|
|
||||||
```
|
|
||||||
|
|
||||||
- Add a new bookmark
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
javascript:(function(){var s=document.createElement('script');s.src=`http://localhost:5174/page-agent.demo.js?t=${Math.random()}`;s.onload=()=>console.log(%27PageAgent ready!%27);document.head.appendChild(s);})();
|
|
||||||
```
|
|
||||||
|
|
||||||
- Click the bookmark on any page to load Page-Agent
|
|
||||||
|
|
||||||
> Warning: AK in your local `.env` will be inlined in the iife script. Be very careful when you distribute the script.
|
|
||||||
|
|
||||||
### Adding Documentation
|
|
||||||
|
|
||||||
Ask an AI to help you add documentation to the `website/` package. Follow the existing style.
|
|
||||||
|
|
||||||
> Our AGENTS.md file and guardrails are designed for this purpose. But please be careful and review anything AI generated.
|
|
||||||
|
|
||||||
## 🚫 What We Don't Accept
|
## 🚫 What We Don't Accept
|
||||||
|
|
||||||
- Breaking changes and large PRs without prior discussion
|
- Breaking changes and large PRs without prior discussion
|
||||||
- Heavy dependencies to core libs
|
- Heavy dependencies to core libs
|
||||||
- Contributions without proper testing
|
|
||||||
- Code that doesn't follow project conventions
|
|
||||||
- Dependencies or code with licenses incompatible with MIT
|
- Dependencies or code with licenses incompatible with MIT
|
||||||
- Bot or AI-generated pull requests without meaningful human involvement
|
- Bot or AI-generated pull requests without meaningful human involvement
|
||||||
|
|
||||||
@@ -194,12 +42,6 @@ Ask an AI to help you add documentation to the `website/` package. Follow the ex
|
|||||||
|
|
||||||
By contributing to this project, you agree that your contributions will be licensed under the MIT License.
|
By contributing to this project, you agree that your contributions will be licensed under the MIT License.
|
||||||
|
|
||||||
> CLA is optional.
|
---
|
||||||
|
|
||||||
## 💬 Questions?
|
|
||||||
|
|
||||||
- Open a GitHub issue for technical questions
|
|
||||||
- Check existing documentation and issues first
|
|
||||||
- Be respectful and constructive in discussions
|
|
||||||
|
|
||||||
Thank you for helping make PageAgent better! 🎉
|
Thank you for helping make PageAgent better! 🎉
|
||||||
|
|||||||
@@ -26,14 +26,15 @@ The GUI Agent Living in Your Webpage. Control web interfaces with natural langua
|
|||||||
- No screenshots. No multi-modal LLMs or special permissions needed.
|
- No screenshots. No multi-modal LLMs or special permissions needed.
|
||||||
- **🧠 Bring your own LLMs**
|
- **🧠 Bring your own LLMs**
|
||||||
- **🐙 Optional [chrome extension](https://alibaba.github.io/page-agent/docs/features/chrome-extension) for multi-page tasks.**
|
- **🐙 Optional [chrome extension](https://alibaba.github.io/page-agent/docs/features/chrome-extension) for multi-page tasks.**
|
||||||
- [MCP Server (Beta)](https://alibaba.github.io/page-agent/docs/features/mcp-server)
|
- And an [MCP Server (Beta)](https://alibaba.github.io/page-agent/docs/features/mcp-server) to control it from outside
|
||||||
|
|
||||||
## 💡 Use Cases
|
## 💡 Use Cases
|
||||||
|
|
||||||
- **SaaS AI Copilot** — Ship an AI copilot in your product in lines of code. No backend rewrite.
|
- **SaaS AI Copilot** — Ship an AI copilot in your product in lines of code. No backend rewrite.
|
||||||
- **Smart Form Filling** — Turn 20-click workflows into one sentence. Perfect for ERP, CRM, and admin systems.
|
- **Smart Form Filling** — Turn 20-click workflows into one sentence. Perfect for ERP, CRM, and admin systems.
|
||||||
- **Accessibility** — Make any web app accessible through natural language. Voice commands, screen readers, zero barrier.
|
- **Accessibility** — Make any web app accessible through natural language. Voice commands, screen readers, zero barrier.
|
||||||
- **Multi-page Agent** — Extend your own agent's reach across browser tabs with the optional [chrome extension](https://alibaba.github.io/page-agent/docs/features/chrome-extension).
|
- **Multi-page Agent** — Extend your own web agent's reach across browser tabs [chrome extension](https://alibaba.github.io/page-agent/docs/features/chrome-extension).
|
||||||
|
- **MCP** - Allow your agent clients to control your browser.
|
||||||
|
|
||||||
## 🚀 Quick Start
|
## 🚀 Quick Start
|
||||||
|
|
||||||
@@ -49,8 +50,8 @@ Fastest way to try PageAgent with our free Demo LLM:
|
|||||||
|
|
||||||
| Mirrors | URL |
|
| Mirrors | URL |
|
||||||
| ------- | ---------------------------------------------------------------------------------- |
|
| ------- | ---------------------------------------------------------------------------------- |
|
||||||
| Global | https://cdn.jsdelivr.net/npm/page-agent@1.6.1/dist/iife/page-agent.demo.js |
|
| Global | https://cdn.jsdelivr.net/npm/page-agent@1.7.1/dist/iife/page-agent.demo.js |
|
||||||
| China | https://registry.npmmirror.com/page-agent/1.6.1/files/dist/iife/page-agent.demo.js |
|
| China | https://registry.npmmirror.com/page-agent/1.7.1/files/dist/iife/page-agent.demo.js |
|
||||||
|
|
||||||
### NPM Installation
|
### NPM Installation
|
||||||
|
|
||||||
@@ -73,13 +74,27 @@ await agent.execute('Click the login button')
|
|||||||
|
|
||||||
For more programmatic usage, see [📖 Documentations](https://alibaba.github.io/page-agent/docs/introduction/overview).
|
For more programmatic usage, see [📖 Documentations](https://alibaba.github.io/page-agent/docs/introduction/overview).
|
||||||
|
|
||||||
|
## 🌟 Awesome Page Agent
|
||||||
|
|
||||||
|
Built something cool with PageAgent? Add it here! Open a PR to share your project.
|
||||||
|
|
||||||
|
> These are community projects — not maintained or endorsed by us. Use at your own discretion.
|
||||||
|
|
||||||
|
| Project | Description |
|
||||||
|
| -------- | ----------------------------------------------------------- |
|
||||||
|
| _Yours?_ | [Open a PR](https://github.com/alibaba/page-agent/pulls) 🙌 |
|
||||||
|
|
||||||
## 🤝 Contributing
|
## 🤝 Contributing
|
||||||
|
|
||||||
We welcome contributions from the community! Follow our instructions in [CONTRIBUTING.md](CONTRIBUTING.md) for setup and guidelines.
|
We welcome contributions from the community! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines and [docs/developer-guide.md](docs/developer-guide.md) for local development workflows.
|
||||||
|
|
||||||
Please read [Code of Conduct](docs/CODE_OF_CONDUCT.md) before contributing.
|
Please read the [maintainer's note](https://github.com/alibaba/page-agent/issues/349) on principles and current state.
|
||||||
|
|
||||||
Contributions generated entirely by bots or agents without substantial human involvement will not be accepted, and bot accounts may be blocked.
|
Contributions generated entirely by **bots or AI** without substantial human involvement will **not be accepted**.
|
||||||
|
|
||||||
|
## ⚖️ License
|
||||||
|
|
||||||
|
[MIT License](LICENSE)
|
||||||
|
|
||||||
## 👏 Acknowledgments
|
## 👏 Acknowledgments
|
||||||
|
|
||||||
@@ -97,23 +112,8 @@ Licensed under the MIT License
|
|||||||
We gratefully acknowledge the browser-use project and its contributors for their
|
We gratefully acknowledge the browser-use project and its contributors for their
|
||||||
excellent work on web automation and DOM interaction patterns that helped make
|
excellent work on web automation and DOM interaction patterns that helped make
|
||||||
this project possible.
|
this project possible.
|
||||||
|
|
||||||
Third-party dependencies and their licenses can be found in the package.json
|
|
||||||
file and in the node_modules directory after installation.
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## 📄 License
|
|
||||||
|
|
||||||
[MIT License](LICENSE)
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
**⭐ Star this repo if you find PageAgent helpful!**
|
**⭐ Star this repo if you find PageAgent helpful!**
|
||||||
|
|
||||||
<a href="https://www.star-history.com/?repos=alibaba%2Fpage-agent&type=date">
|
|
||||||
<picture>
|
|
||||||
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/image?repos=alibaba/page-agent&type=date&theme=dark&legend=top-left&v=7" />
|
|
||||||
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/image?repos=alibaba/page-agent&type=date&legend=top-left&v=7" />
|
|
||||||
<img alt="Star History Chart" src="https://api.star-history.com/image?repos=alibaba/page-agent&type=date&legend=top-left&v=7" />
|
|
||||||
</picture>
|
|
||||||
</a>
|
|
||||||
|
|||||||
+10
-17
@@ -32,7 +32,8 @@
|
|||||||
- **SaaS AI Copilot** — 几行代码为你的产品加上 AI 副驾驶,无需重写后端。
|
- **SaaS AI Copilot** — 几行代码为你的产品加上 AI 副驾驶,无需重写后端。
|
||||||
- **智能表单填写** — 把 20 次点击变成一句话。ERP、CRM、管理后台的最佳拍档。
|
- **智能表单填写** — 把 20 次点击变成一句话。ERP、CRM、管理后台的最佳拍档。
|
||||||
- **无障碍增强** — 用自然语言让任何网页无障碍。语音指令、屏幕阅读器,零门槛。
|
- **无障碍增强** — 用自然语言让任何网页无障碍。语音指令、屏幕阅读器,零门槛。
|
||||||
- **跨页面 Agent** — 通过可选的 [Chrome 扩展](https://alibaba.github.io/page-agent/docs/features/chrome-extension),让你自己的 Agent 跨标签页工作。
|
- **跨页面 Agent** — 通过可选的 [Chrome 扩展](https://alibaba.github.io/page-agent/docs/features/chrome-extension),让你自己的 Web Agent 跨标签页工作。
|
||||||
|
- 通过 MCP 为现有 Agent 加入浏览器控制能力。
|
||||||
|
|
||||||
## 🚀 快速开始
|
## 🚀 快速开始
|
||||||
|
|
||||||
@@ -48,8 +49,8 @@
|
|||||||
|
|
||||||
| Mirrors | URL |
|
| Mirrors | URL |
|
||||||
| ------- | ---------------------------------------------------------------------------------- |
|
| ------- | ---------------------------------------------------------------------------------- |
|
||||||
| Global | https://cdn.jsdelivr.net/npm/page-agent@1.6.1/dist/iife/page-agent.demo.js |
|
| Global | https://cdn.jsdelivr.net/npm/page-agent@1.7.1/dist/iife/page-agent.demo.js |
|
||||||
| China | https://registry.npmmirror.com/page-agent/1.6.1/files/dist/iife/page-agent.demo.js |
|
| China | https://registry.npmmirror.com/page-agent/1.7.1/files/dist/iife/page-agent.demo.js |
|
||||||
|
|
||||||
### NPM 安装
|
### NPM 安装
|
||||||
|
|
||||||
@@ -74,11 +75,13 @@ await agent.execute('点击登录按钮')
|
|||||||
|
|
||||||
## 🤝 贡献
|
## 🤝 贡献
|
||||||
|
|
||||||
欢迎社区贡献!请参阅 [CONTRIBUTING.md](../CONTRIBUTING.md) 了解安装与贡献指南。请在贡献前阅读[行为准则](CODE_OF_CONDUCT.md)。
|
欢迎社区贡献!请参阅 [CONTRIBUTING.md](../CONTRIBUTING.md) 了解安装与贡献指南。
|
||||||
|
|
||||||
我们不接受未经实质性人类参与、完全由 Bot 或 Agent 自动生成的代码,机器人账号可能被禁止参与互动。
|
提交 issue 或 PR 之前,请先阅读[作者声明](https://github.com/alibaba/page-agent/issues/349)和[行为准则](CODE_OF_CONDUCT.md)。
|
||||||
|
|
||||||
## 👏 致谢
|
我们不接受未经实质性人类参与、完全由 Bot 或 Agent 自动生成的代码。
|
||||||
|
|
||||||
|
## 👏 声明与致谢
|
||||||
|
|
||||||
本项目基于 **[`browser-use`](https://github.com/browser-use/browser-use)** 的优秀工作构建。
|
本项目基于 **[`browser-use`](https://github.com/browser-use/browser-use)** 的优秀工作构建。
|
||||||
|
|
||||||
@@ -94,12 +97,9 @@ Licensed under the MIT License
|
|||||||
We gratefully acknowledge the browser-use project and its contributors for their
|
We gratefully acknowledge the browser-use project and its contributors for their
|
||||||
excellent work on web automation and DOM interaction patterns that helped make
|
excellent work on web automation and DOM interaction patterns that helped make
|
||||||
this project possible.
|
this project possible.
|
||||||
|
|
||||||
Third-party dependencies and their licenses can be found in the package.json
|
|
||||||
file and in the node_modules directory after installation.
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## 📄 许可证
|
## ⚖️ 许可证
|
||||||
|
|
||||||
[MIT License](../LICENSE)
|
[MIT License](../LICENSE)
|
||||||
|
|
||||||
@@ -107,10 +107,3 @@ file and in the node_modules directory after installation.
|
|||||||
|
|
||||||
**⭐ 如果觉得 PageAgent 有用或有趣,请给项目点个星!**
|
**⭐ 如果觉得 PageAgent 有用或有趣,请给项目点个星!**
|
||||||
|
|
||||||
<a href="https://www.star-history.com/?repos=alibaba%2Fpage-agent&type=date">
|
|
||||||
<picture>
|
|
||||||
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/image?repos=alibaba/page-agent&type=date&theme=dark&legend=top-left&v=7" />
|
|
||||||
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/image?repos=alibaba/page-agent&type=date&legend=top-left&v=7" />
|
|
||||||
<img alt="Star History Chart" src="https://api.star-history.com/image?repos=alibaba/page-agent&type=date&legend=top-left&v=7" />
|
|
||||||
</picture>
|
|
||||||
</a>
|
|
||||||
|
|||||||
@@ -0,0 +1,117 @@
|
|||||||
|
# Developer Guide
|
||||||
|
|
||||||
|
This file is for local development workflows.
|
||||||
|
|
||||||
|
For contribution rules and expectations, see [../CONTRIBUTING.md](../CONTRIBUTING.md).
|
||||||
|
|
||||||
|
## 🚀 Quick Start
|
||||||
|
|
||||||
|
### Development Setup
|
||||||
|
|
||||||
|
1. **Prerequisites**
|
||||||
|
- `macOS` / `Linux` / `WSL`
|
||||||
|
- `node.js >= 20` with `npm >= 10`
|
||||||
|
- An editor that supports `ts/eslint/prettier`
|
||||||
|
- Make sure `eslint`, `prettier` and `commitlint` work well. Un-linted code won't pass the CI.
|
||||||
|
|
||||||
|
2. **Setup**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm i
|
||||||
|
npm start # Start demo and documentation site
|
||||||
|
npm run build # Build libs and website
|
||||||
|
```
|
||||||
|
|
||||||
|
## 📦 Project Structure
|
||||||
|
|
||||||
|
This is a **monorepo** with npm workspaces containing **4 main packages**:
|
||||||
|
|
||||||
|
- **Page Agent** (`packages/page-agent/`) - Main entry with built-in UI Panel, published as `page-agent` on npm
|
||||||
|
- **Core** (`packages/core/`) - Core agent logic without UI (npm: `@page-agent/core`)
|
||||||
|
- **Extension** (`packages/extension/`) - Chrome extension for multi-page tasks and browser-level automation
|
||||||
|
- **Website** (`packages/website/`) - React documentation and landing page. Also as demo and test page for the core lib. private package `@page-agent/website`
|
||||||
|
|
||||||
|
> We use a simplified monorepo solution with `native npm-workspace + ts reference + vite alias`. No fancy tooling. Hoisting is required.
|
||||||
|
>
|
||||||
|
> - When developing. Use alias so that we don't have to pre-build.
|
||||||
|
> - When bundling. Use external and disable ts `paths` alias.
|
||||||
|
> - When bundling `IIFE` and `Website`. Bundle everything together.
|
||||||
|
|
||||||
|
## 🤖 AGENTS.md Alias
|
||||||
|
|
||||||
|
If your AI assistant does not support [AGENTS.md](https://agents.md/). Add an alias for it:
|
||||||
|
|
||||||
|
- claude-code (`CLAUDE.md`)
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
@AGENTS.md
|
||||||
|
```
|
||||||
|
|
||||||
|
- antigravity (`.agent/rules/alias.md`)
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
---
|
||||||
|
trigger: always_on
|
||||||
|
---
|
||||||
|
|
||||||
|
@../../AGENTS.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🔧 Development Workflows
|
||||||
|
|
||||||
|
### Test With Your Own LLM API
|
||||||
|
|
||||||
|
- Create a `.env` file in the repo root with your LLM API config
|
||||||
|
|
||||||
|
```env
|
||||||
|
LLM_MODEL_NAME=gpt-5.2
|
||||||
|
LLM_API_KEY=your-api-key
|
||||||
|
LLM_BASE_URL=https://api.your-llm-provider.com/v1
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Ollama example** (tested on 0.15 + qwen3:14b, RTX3090 24GB):
|
||||||
|
|
||||||
|
```env
|
||||||
|
LLM_BASE_URL="http://localhost:11434/v1"
|
||||||
|
LLM_API_KEY="NA"
|
||||||
|
LLM_MODEL_NAME="qwen3:14b"
|
||||||
|
```
|
||||||
|
|
||||||
|
> @see https://alibaba.github.io/page-agent/docs/features/models#ollama for configuration
|
||||||
|
|
||||||
|
- **Restart the dev server** to load new env vars
|
||||||
|
- If not provided, the demo will use the free testing proxy by default. By using it, you agree to its [terms](./terms-and-privacy.md).
|
||||||
|
|
||||||
|
### Extension Development
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# make sure you ran `npm run build:libs` first and every time you changed the core libs
|
||||||
|
npm run dev -w @page-agent/ext
|
||||||
|
npm run zip -w @page-agent/ext
|
||||||
|
```
|
||||||
|
|
||||||
|
- Update `packages/extension/docs/extension_api.md` for API integration details
|
||||||
|
|
||||||
|
### Testing on Other Websites
|
||||||
|
|
||||||
|
- Start and serve a local `iife` script
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm run dev:demo # Serving IIFE with auto rebuild at http://localhost:5174/page-agent.demo.js
|
||||||
|
```
|
||||||
|
|
||||||
|
- Add a new bookmark
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
javascript:(function(){var s=document.createElement('script');s.src=`http://localhost:5174/page-agent.demo.js?t=${Math.random()}`;s.onload=()=>console.log(%27PageAgent ready!%27);document.head.appendChild(s);})();
|
||||||
|
```
|
||||||
|
|
||||||
|
- Click the bookmark on any page to load Page-Agent
|
||||||
|
|
||||||
|
> Warning: AK in your local `.env` will be inlined in the iife script. Be very careful when you distribute the script.
|
||||||
|
|
||||||
|
### Adding Documentation
|
||||||
|
|
||||||
|
Ask an AI to help you add documentation to the `website/` package. Follow the existing style.
|
||||||
|
|
||||||
|
> Our AGENTS.md file and guardrails are designed for this purpose. But please be careful and review anything AI generated.
|
||||||
Generated
+260
-260
File diff suppressed because it is too large
Load Diff
+3
-3
@@ -1,7 +1,7 @@
|
|||||||
{
|
{
|
||||||
"name": "root",
|
"name": "root",
|
||||||
"private": true,
|
"private": true,
|
||||||
"version": "1.6.1",
|
"version": "1.7.1",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"workspaces": [
|
"workspaces": [
|
||||||
"packages/page-controller",
|
"packages/page-controller",
|
||||||
@@ -42,7 +42,7 @@
|
|||||||
"@commitlint/config-conventional": "^20.5.0",
|
"@commitlint/config-conventional": "^20.5.0",
|
||||||
"@eslint/js": "^9.39.2",
|
"@eslint/js": "^9.39.2",
|
||||||
"@microsoft/api-extractor": "^7.57.7",
|
"@microsoft/api-extractor": "^7.57.7",
|
||||||
"@tailwindcss/vite": "^4.2.1",
|
"@tailwindcss/vite": "^4.2.2",
|
||||||
"@trivago/prettier-plugin-sort-imports": "^6.0.2",
|
"@trivago/prettier-plugin-sort-imports": "^6.0.2",
|
||||||
"@types/node": "^25.5.0",
|
"@types/node": "^25.5.0",
|
||||||
"@vitejs/plugin-react-swc": "^4.3.0",
|
"@vitejs/plugin-react-swc": "^4.3.0",
|
||||||
@@ -60,7 +60,7 @@
|
|||||||
"lint-staged": "^16.4.0",
|
"lint-staged": "^16.4.0",
|
||||||
"prettier": "^3.8.0",
|
"prettier": "^3.8.0",
|
||||||
"typescript": "^5.9.3",
|
"typescript": "^5.9.3",
|
||||||
"typescript-eslint": "^8.57.1",
|
"typescript-eslint": "^8.58.0",
|
||||||
"unplugin-dts": "^1.0.0-beta.6",
|
"unplugin-dts": "^1.0.0-beta.6",
|
||||||
"vite": "^7.3.1",
|
"vite": "^7.3.1",
|
||||||
"vite-plugin-css-injected-by-js": "^4.0.1",
|
"vite-plugin-css-injected-by-js": "^4.0.1",
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
{
|
{
|
||||||
"name": "@page-agent/core",
|
"name": "@page-agent/core",
|
||||||
"private": false,
|
"private": false,
|
||||||
"version": "1.6.1",
|
"version": "1.7.1",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "./dist/esm/page-agent-core.js",
|
"main": "./dist/esm/page-agent-core.js",
|
||||||
"module": "./dist/esm/page-agent-core.js",
|
"module": "./dist/esm/page-agent-core.js",
|
||||||
@@ -44,8 +44,8 @@
|
|||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"chalk": "^5.6.2",
|
"chalk": "^5.6.2",
|
||||||
"@page-agent/llms": "1.6.1",
|
"@page-agent/llms": "1.7.1",
|
||||||
"@page-agent/page-controller": "1.6.1"
|
"@page-agent/page-controller": "1.7.1"
|
||||||
},
|
},
|
||||||
"peerDependencies": {
|
"peerDependencies": {
|
||||||
"zod": "^3.25.0 || ^4.0.0"
|
"zod": "^3.25.0 || ^4.0.0"
|
||||||
|
|||||||
@@ -131,7 +131,8 @@ tools.set(
|
|||||||
tools.set(
|
tools.set(
|
||||||
'scroll',
|
'scroll',
|
||||||
tool({
|
tool({
|
||||||
description: 'Scroll the page vertically. Use index for scroll elements (dropdowns/custom UI).',
|
description:
|
||||||
|
'Scroll vertically. Without index: scrolls the document. With index: scrolls the container at that index (or its nearest scrollable ancestor). Use index of a data-scrollable element to scroll a specific area.',
|
||||||
inputSchema: z.object({
|
inputSchema: z.object({
|
||||||
down: z.boolean().default(true),
|
down: z.boolean().default(true),
|
||||||
num_pages: z.number().min(0).max(10).optional().default(0.1),
|
num_pages: z.number().min(0).max(10).optional().default(0.1),
|
||||||
@@ -155,7 +156,7 @@ tools.set(
|
|||||||
'scroll_horizontally',
|
'scroll_horizontally',
|
||||||
tool({
|
tool({
|
||||||
description:
|
description:
|
||||||
'Scroll the page horizontally, or within a specific element by index. Useful for wide tables.',
|
'Scroll horizontally. Without index: scrolls the document. With index: scrolls the container at that index (or its nearest scrollable ancestor). Use index of a data-scrollable element to scroll a specific area.',
|
||||||
inputSchema: z.object({
|
inputSchema: z.object({
|
||||||
right: z.boolean().default(true),
|
right: z.boolean().default(true),
|
||||||
pixels: z.number().int().min(0),
|
pixels: z.number().int().min(0),
|
||||||
|
|||||||
@@ -118,9 +118,18 @@ export interface ExecuteConfig {
|
|||||||
model: string
|
model: string
|
||||||
apiKey?: string
|
apiKey?: string
|
||||||
|
|
||||||
|
// Global system-level instructions for the agent.
|
||||||
|
// Equivalent to AgentConfig.instructions.system.
|
||||||
|
systemInstruction?: string
|
||||||
|
|
||||||
// Include the initial tab where page JS starts. Default: true.
|
// Include the initial tab where page JS starts. Default: true.
|
||||||
includeInitialTab?: boolean
|
includeInitialTab?: boolean
|
||||||
|
|
||||||
|
// Control all unpinned tabs in the window instead of only the tab group.
|
||||||
|
// When enabled, agent sees and can switch to every non-pinned tab.
|
||||||
|
// Default: false. Experimental.
|
||||||
|
experimentalIncludeAllTabs?: boolean
|
||||||
|
|
||||||
onStatusChange?: (status: AgentStatus) => void
|
onStatusChange?: (status: AgentStatus) => void
|
||||||
onActivity?: (activity: AgentActivity) => void
|
onActivity?: (activity: AgentActivity) => void
|
||||||
onHistoryUpdate?: (history: HistoricalEvent[]) => void
|
onHistoryUpdate?: (history: HistoricalEvent[]) => void
|
||||||
@@ -207,7 +216,11 @@ interface ExecuteConfig {
|
|||||||
baseURL: string
|
baseURL: string
|
||||||
model: string
|
model: string
|
||||||
apiKey?: string
|
apiKey?: string
|
||||||
|
|
||||||
|
systemInstruction?: string
|
||||||
|
|
||||||
includeInitialTab?: boolean
|
includeInitialTab?: boolean
|
||||||
|
experimentalIncludeAllTabs?: boolean
|
||||||
onStatusChange?: (status: AgentStatus) => void
|
onStatusChange?: (status: AgentStatus) => void
|
||||||
onActivity?: (activity: AgentActivity) => void
|
onActivity?: (activity: AgentActivity) => void
|
||||||
onHistoryUpdate?: (history: HistoricalEvent[]) => void
|
onHistoryUpdate?: (history: HistoricalEvent[]) => void
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
{
|
{
|
||||||
"name": "@page-agent/ext",
|
"name": "@page-agent/ext",
|
||||||
"private": true,
|
"private": true,
|
||||||
"version": "1.6.1",
|
"version": "1.7.1",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"dev": "wxt",
|
"dev": "wxt",
|
||||||
@@ -16,31 +16,31 @@
|
|||||||
"@radix-ui/react-separator": "^1.1.8",
|
"@radix-ui/react-separator": "^1.1.8",
|
||||||
"@radix-ui/react-slot": "^1.2.4",
|
"@radix-ui/react-slot": "^1.2.4",
|
||||||
"@radix-ui/react-switch": "^1.2.6",
|
"@radix-ui/react-switch": "^1.2.6",
|
||||||
"@types/chrome": "^0.1.37",
|
"@types/chrome": "^0.1.38",
|
||||||
"@types/react": "^19.2.14",
|
"@types/react": "^19.2.14",
|
||||||
"@types/react-dom": "^19.2.1",
|
"@types/react-dom": "^19.2.1",
|
||||||
"@wxt-dev/module-react": "^1.2.2",
|
"@wxt-dev/module-react": "^1.2.2",
|
||||||
"class-variance-authority": "^0.7.1",
|
"class-variance-authority": "^0.7.1",
|
||||||
"clsx": "^2.1.1",
|
"clsx": "^2.1.1",
|
||||||
"idb": "^8.0.3",
|
"idb": "^8.0.3",
|
||||||
"lucide-react": "^0.577.0",
|
"lucide-react": "^1.7.0",
|
||||||
"motion": "^12.37.0",
|
"motion": "^12.38.0",
|
||||||
"next-themes": "^0.4.6",
|
"next-themes": "^0.4.6",
|
||||||
"react": "^19.2.4",
|
"react": "^19.2.4",
|
||||||
"react-dom": "^19.2.4",
|
"react-dom": "^19.2.4",
|
||||||
"rough-notation": "^0.5.1",
|
"rough-notation": "^0.5.1",
|
||||||
"simple-icons": "^16.12.0",
|
"simple-icons": "^16.14.0",
|
||||||
"sonner": "^2.0.7",
|
"sonner": "^2.0.7",
|
||||||
"tailwind-merge": "^3.5.0",
|
"tailwind-merge": "^3.5.0",
|
||||||
"tailwindcss": "^4.1.14",
|
"tailwindcss": "^4.1.14",
|
||||||
"tw-animate-css": "^1.4.0",
|
"tw-animate-css": "^1.4.0",
|
||||||
"wxt": "^0.20.19"
|
"wxt": "^0.20.20"
|
||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@page-agent/core": "1.6.1",
|
"@page-agent/core": "1.7.1",
|
||||||
"@page-agent/llms": "1.6.1",
|
"@page-agent/llms": "1.7.1",
|
||||||
"@page-agent/page-controller": "1.6.1",
|
"@page-agent/page-controller": "1.7.1",
|
||||||
"@page-agent/ui": "1.6.1",
|
"@page-agent/ui": "1.7.1",
|
||||||
"ai-motion": "^0.4.8",
|
"ai-motion": "^0.4.8",
|
||||||
"chalk": "^5.6.2"
|
"chalk": "^5.6.2"
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -11,13 +11,18 @@ function detectLanguage(): 'en-US' | 'zh-CN' {
|
|||||||
return lang.startsWith('zh') ? 'zh-CN' : 'en-US'
|
return lang.startsWith('zh') ? 'zh-CN' : 'en-US'
|
||||||
}
|
}
|
||||||
|
|
||||||
|
interface MultiPageAgentConfig extends AgentConfig {
|
||||||
|
includeInitialTab?: boolean
|
||||||
|
experimentalIncludeAllTabs?: boolean
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* MultiPageAgent
|
* MultiPageAgent
|
||||||
* - use with extension
|
* - use with extension
|
||||||
* - can be used from a side panel or a content script
|
* - can be used from a side panel or a content script
|
||||||
*/
|
*/
|
||||||
export class MultiPageAgent extends PageAgentCore {
|
export class MultiPageAgent extends PageAgentCore {
|
||||||
constructor(config: AgentConfig & { includeInitialTab?: boolean }) {
|
constructor(config: MultiPageAgentConfig) {
|
||||||
// multi page controller
|
// multi page controller
|
||||||
const tabsController = new TabsController()
|
const tabsController = new TabsController()
|
||||||
const pageController = new RemotePageController(tabsController)
|
const pageController = new RemotePageController(tabsController)
|
||||||
@@ -31,8 +36,8 @@ export class MultiPageAgent extends PageAgentCore {
|
|||||||
`Default working language: **${targetLanguage}**`
|
`Default working language: **${targetLanguage}**`
|
||||||
)
|
)
|
||||||
|
|
||||||
// include initial tab for controlling
|
|
||||||
const includeInitialTab = config.includeInitialTab ?? true
|
const includeInitialTab = config.includeInitialTab ?? true
|
||||||
|
const experimentalIncludeAllTabs = config.experimentalIncludeAllTabs ?? false
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* When the agent is in side-panel and user closed the side-panel.
|
* When the agent is in side-panel and user closed the side-panel.
|
||||||
@@ -50,7 +55,7 @@ export class MultiPageAgent extends PageAgentCore {
|
|||||||
customSystemPrompt: systemPrompt,
|
customSystemPrompt: systemPrompt,
|
||||||
|
|
||||||
onBeforeTask: async (agent) => {
|
onBeforeTask: async (agent) => {
|
||||||
await tabsController.init(agent.task, includeInitialTab)
|
await tabsController.init(agent.task, { includeInitialTab, experimentalIncludeAllTabs })
|
||||||
|
|
||||||
heartBeatInterval = window.setInterval(() => {
|
heartBeatInterval = window.setInterval(() => {
|
||||||
chrome.storage.local.set({
|
chrome.storage.local.set({
|
||||||
|
|||||||
@@ -10,9 +10,7 @@ export function handlePageControlMessage(
|
|||||||
): true | undefined {
|
): true | undefined {
|
||||||
const PREFIX = '[RemotePageController.background]'
|
const PREFIX = '[RemotePageController.background]'
|
||||||
|
|
||||||
function debug(...messages: any[]) {
|
const debug = console.debug.bind(console, `\x1b[90m${PREFIX}\x1b[0m`)
|
||||||
console.debug(`\x1b[90m${PREFIX}\x1b[0m`, ...messages)
|
|
||||||
}
|
|
||||||
|
|
||||||
const { action, payload, targetTabId } = message
|
const { action, payload, targetTabId } = message
|
||||||
|
|
||||||
|
|||||||
@@ -19,7 +19,10 @@ export function initPageController() {
|
|||||||
|
|
||||||
function getPC(): PageController {
|
function getPC(): PageController {
|
||||||
if (!pageController) {
|
if (!pageController) {
|
||||||
pageController = new PageController({ enableMask: false, viewportExpansion: 400 })
|
pageController = new PageController({
|
||||||
|
enableMask: false,
|
||||||
|
viewportExpansion: 400,
|
||||||
|
})
|
||||||
}
|
}
|
||||||
return pageController
|
return pageController
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -4,9 +4,7 @@ import type { TabsController } from './TabsController'
|
|||||||
|
|
||||||
const PREFIX = '[RemotePageController]'
|
const PREFIX = '[RemotePageController]'
|
||||||
|
|
||||||
function debug(...messages: any[]) {
|
const debug = console.debug.bind(console, `\x1b[90m${PREFIX}\x1b[0m`)
|
||||||
console.debug(`\x1b[90m${PREFIX}\x1b[0m`, ...messages)
|
|
||||||
}
|
|
||||||
|
|
||||||
function sendMessage(message: {
|
function sendMessage(message: {
|
||||||
type: 'PAGE_CONTROL'
|
type: 'PAGE_CONTROL'
|
||||||
|
|||||||
@@ -5,9 +5,7 @@ import type { TabAction } from './TabsController'
|
|||||||
|
|
||||||
const PREFIX = '[TabsController.background]'
|
const PREFIX = '[TabsController.background]'
|
||||||
|
|
||||||
function debug(...messages: any[]) {
|
const debug = console.debug.bind(console, `\x1b[90m${PREFIX}\x1b[0m`)
|
||||||
console.debug(`\x1b[90m${PREFIX}\x1b[0m`, ...messages)
|
|
||||||
}
|
|
||||||
|
|
||||||
export function handleTabControlMessage(
|
export function handleTabControlMessage(
|
||||||
message: { type: 'TAB_CONTROL'; action: TabAction; payload: any },
|
message: { type: 'TAB_CONTROL'; action: TabAction; payload: any },
|
||||||
@@ -20,11 +18,10 @@ export function handleTabControlMessage(
|
|||||||
case 'get_active_tab': {
|
case 'get_active_tab': {
|
||||||
debug('get_active_tab')
|
debug('get_active_tab')
|
||||||
chrome.tabs
|
chrome.tabs
|
||||||
.query({ active: true, currentWindow: true })
|
.query({ active: true })
|
||||||
.then((tabs) => {
|
.then((tabs) => {
|
||||||
const tabId = tabs.length > 0 ? tabs[0].id || null : null
|
debug('get_active_tab: success', tabs)
|
||||||
debug('get_active_tab: success', tabId)
|
sendResponse({ success: true, tab: tabs[0] })
|
||||||
sendResponse({ success: true, tabId })
|
|
||||||
})
|
})
|
||||||
.catch((error) => {
|
.catch((error) => {
|
||||||
sendResponse({ error: error instanceof Error ? error.message : String(error) })
|
sendResponse({ error: error instanceof Error ? error.message : String(error) })
|
||||||
@@ -63,7 +60,7 @@ export function handleTabControlMessage(
|
|||||||
case 'create_tab_group': {
|
case 'create_tab_group': {
|
||||||
debug('create_tab_group', payload)
|
debug('create_tab_group', payload)
|
||||||
chrome.tabs
|
chrome.tabs
|
||||||
.group({ tabIds: payload.tabIds })
|
.group({ tabIds: payload.tabIds, createProperties: { windowId: payload.windowId } })
|
||||||
.then((groupId) => {
|
.then((groupId) => {
|
||||||
debug('create_tab_group: success', groupId)
|
debug('create_tab_group: success', groupId)
|
||||||
sendResponse({ success: true, groupId })
|
sendResponse({ success: true, groupId })
|
||||||
@@ -114,47 +111,59 @@ export function handleTabControlMessage(
|
|||||||
return true // async response
|
return true // async response
|
||||||
}
|
}
|
||||||
|
|
||||||
|
case 'get_window_tabs': {
|
||||||
|
debug('get_window_tabs', payload)
|
||||||
|
chrome.tabs
|
||||||
|
.query({ windowId: payload.windowId })
|
||||||
|
.then((tabs) => {
|
||||||
|
sendResponse({ success: true, tabs })
|
||||||
|
})
|
||||||
|
.catch((error) => {
|
||||||
|
sendResponse({ error: error instanceof Error ? error.message : String(error) })
|
||||||
|
})
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
default:
|
default:
|
||||||
sendResponse({ error: `Unknown action: ${action}` })
|
sendResponse({ error: `Unknown action: ${action}` })
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
export function setupTabChangeEvents() {
|
const tabEventPorts = new Set<chrome.runtime.Port>()
|
||||||
console.log('[TabsController.background] setupTabChangeEvents')
|
|
||||||
|
function broadcastTabEvent(message: object) {
|
||||||
|
for (const port of tabEventPorts) {
|
||||||
|
port.postMessage(message)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Port-based tab events: agents connect via `chrome.runtime.connect({ name: 'tab-events' })`
|
||||||
|
* and receive tab change events through the port. Works for both extension pages and content scripts.
|
||||||
|
*/
|
||||||
|
export function setupTabEventsPort() {
|
||||||
|
chrome.runtime.onConnect.addListener((port) => {
|
||||||
|
if (port.name !== 'tab-events') return
|
||||||
|
|
||||||
|
debug('port connected', port.sender?.tab?.id ?? port.sender?.url)
|
||||||
|
tabEventPorts.add(port)
|
||||||
|
|
||||||
|
port.onDisconnect.addListener(() => {
|
||||||
|
debug('port disconnected')
|
||||||
|
tabEventPorts.delete(port)
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
chrome.tabs.onCreated.addListener((tab) => {
|
chrome.tabs.onCreated.addListener((tab) => {
|
||||||
debug('onCreated', tab)
|
broadcastTabEvent({ action: 'created', payload: { tab } })
|
||||||
chrome.runtime
|
|
||||||
.sendMessage({ type: 'TAB_CHANGE', action: 'created', payload: { tab } })
|
|
||||||
.catch((error) => {
|
|
||||||
debug('onCreated error:', error)
|
|
||||||
})
|
|
||||||
})
|
})
|
||||||
|
|
||||||
chrome.tabs.onRemoved.addListener((tabId, removeInfo) => {
|
chrome.tabs.onRemoved.addListener((tabId, removeInfo) => {
|
||||||
debug('onRemoved', tabId, removeInfo)
|
broadcastTabEvent({ action: 'removed', payload: { tabId, removeInfo } })
|
||||||
chrome.runtime
|
|
||||||
.sendMessage({
|
|
||||||
type: 'TAB_CHANGE',
|
|
||||||
action: 'removed',
|
|
||||||
payload: { tabId, removeInfo },
|
|
||||||
})
|
|
||||||
.catch((error) => {
|
|
||||||
debug('onRemoved error:', error)
|
|
||||||
})
|
|
||||||
})
|
})
|
||||||
|
|
||||||
chrome.tabs.onUpdated.addListener((tabId, changeInfo, tab) => {
|
chrome.tabs.onUpdated.addListener((tabId, changeInfo, tab) => {
|
||||||
debug('onUpdated', tabId, changeInfo)
|
broadcastTabEvent({ action: 'updated', payload: { tabId, changeInfo, tab } })
|
||||||
chrome.runtime
|
|
||||||
.sendMessage({
|
|
||||||
type: 'TAB_CHANGE',
|
|
||||||
action: 'updated',
|
|
||||||
payload: { tabId, changeInfo, tab },
|
|
||||||
})
|
|
||||||
.catch((error) => {
|
|
||||||
debug('onUpdated error:', error)
|
|
||||||
})
|
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2,9 +2,7 @@ import { isContentScriptAllowed } from './RemotePageController'
|
|||||||
|
|
||||||
const PREFIX = '[TabsController]'
|
const PREFIX = '[TabsController]'
|
||||||
|
|
||||||
function debug(...messages: any[]) {
|
const debug = console.debug.bind(console, `\x1b[90m${PREFIX}\x1b[0m`)
|
||||||
console.debug(`\x1b[90m${PREFIX}\x1b[0m`, ...messages)
|
|
||||||
}
|
|
||||||
|
|
||||||
function sendMessage(message: {
|
function sendMessage(message: {
|
||||||
type: 'TAB_CONTROL'
|
type: 'TAB_CONTROL'
|
||||||
@@ -22,46 +20,91 @@ function sendMessage(message: {
|
|||||||
* - live in the agent env (extension page or content script)
|
* - live in the agent env (extension page or content script)
|
||||||
* - no chrome apis. call sw for tab operations
|
* - no chrome apis. call sw for tab operations
|
||||||
*/
|
*/
|
||||||
export class TabsController extends EventTarget {
|
export class TabsController {
|
||||||
currentTabId: number | null = null
|
currentTabId: number | null = null
|
||||||
|
|
||||||
|
private disposed = false
|
||||||
|
private port: chrome.runtime.Port | null = null
|
||||||
|
private portRetries = 0
|
||||||
|
|
||||||
|
private windowId: number | null = null
|
||||||
private tabs: TabMeta[] = []
|
private tabs: TabMeta[] = []
|
||||||
private initialTabId: number | null = null
|
private initialTabId: number | null = null
|
||||||
private tabGroupId: number | null = null
|
private tabGroupId: number | null = null
|
||||||
|
private experimentalIncludeAllTabs = false
|
||||||
private task: string = ''
|
private task: string = ''
|
||||||
|
|
||||||
async init(task: string, includeInitialTab: boolean = true) {
|
async init(task: string, options: TabsInitOptions = {}) {
|
||||||
debug('init', task, includeInitialTab)
|
const { includeInitialTab = true, experimentalIncludeAllTabs = false } = options
|
||||||
|
debug('init', task, options)
|
||||||
|
|
||||||
|
if (this.disposed) {
|
||||||
|
throw new Error('TabsController already disposed')
|
||||||
|
}
|
||||||
|
|
||||||
this.task = task
|
|
||||||
this.tabs = []
|
|
||||||
this.currentTabId = null
|
this.currentTabId = null
|
||||||
|
this.disposed = false
|
||||||
|
this.port = null
|
||||||
|
this.portRetries = 0
|
||||||
|
|
||||||
|
this.windowId = null
|
||||||
|
this.tabs = []
|
||||||
this.tabGroupId = null
|
this.tabGroupId = null
|
||||||
this.initialTabId = null
|
this.initialTabId = null
|
||||||
|
this.experimentalIncludeAllTabs = experimentalIncludeAllTabs
|
||||||
|
this.task = task
|
||||||
|
|
||||||
const result = await sendMessage({
|
const activeTabResult = await sendMessage({
|
||||||
type: 'TAB_CONTROL',
|
type: 'TAB_CONTROL',
|
||||||
action: 'get_active_tab',
|
action: 'get_active_tab',
|
||||||
})
|
})
|
||||||
|
|
||||||
this.initialTabId = result.tabId
|
this.initialTabId = activeTabResult.tab?.id
|
||||||
|
this.windowId = activeTabResult.tab?.windowId
|
||||||
|
|
||||||
if (!this.initialTabId) {
|
if (!this.initialTabId || !this.windowId) {
|
||||||
throw new Error('Failed to get initial tab ID')
|
if (activeTabResult.error) {
|
||||||
|
throw new Error(activeTabResult.error)
|
||||||
|
} else {
|
||||||
|
throw new Error('Failed to get active tab')
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if (includeInitialTab) {
|
this.connectTabEvents()
|
||||||
|
|
||||||
|
if (experimentalIncludeAllTabs) {
|
||||||
|
const allTabs = await sendMessage({
|
||||||
|
type: 'TAB_CONTROL',
|
||||||
|
action: 'get_window_tabs',
|
||||||
|
payload: { windowId: this.windowId },
|
||||||
|
})
|
||||||
|
for (const tab of allTabs.tabs as chrome.tabs.Tab[]) {
|
||||||
|
if (tab.id && !tab.pinned && isContentScriptAllowed(tab.url)) {
|
||||||
|
this.addTab({
|
||||||
|
id: tab.id,
|
||||||
|
isInitial: tab.id === this.initialTabId,
|
||||||
|
url: tab.url,
|
||||||
|
title: tab.title,
|
||||||
|
status: tab.status,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (this.tabs.find((t) => t.id === this.initialTabId)) {
|
||||||
|
this.currentTabId = this.initialTabId
|
||||||
|
await this.createTabGroup([this.initialTabId])
|
||||||
|
}
|
||||||
|
} else if (includeInitialTab) {
|
||||||
const info = await sendMessage({
|
const info = await sendMessage({
|
||||||
type: 'TAB_CONTROL',
|
type: 'TAB_CONTROL',
|
||||||
action: 'get_tab_info',
|
action: 'get_tab_info',
|
||||||
payload: { tabId: this.initialTabId },
|
payload: { tabId: this.initialTabId },
|
||||||
})
|
})
|
||||||
|
|
||||||
if (isContentScriptAllowed(info.url)) {
|
if (isContentScriptAllowed(info.url) && !info.pinned) {
|
||||||
this.currentTabId = this.initialTabId
|
this.currentTabId = this.initialTabId
|
||||||
|
|
||||||
this.tabs.push({
|
this.addTab({
|
||||||
id: result.tabId,
|
id: this.initialTabId,
|
||||||
isInitial: true,
|
isInitial: true,
|
||||||
url: info.url,
|
url: info.url,
|
||||||
title: info.title,
|
title: info.title,
|
||||||
@@ -73,52 +116,6 @@ export class TabsController extends EventTarget {
|
|||||||
}
|
}
|
||||||
|
|
||||||
await this.updateCurrentTabId(this.currentTabId)
|
await this.updateCurrentTabId(this.currentTabId)
|
||||||
|
|
||||||
const tabChangeHandler = (message: any): void => {
|
|
||||||
if (message.type !== 'TAB_CHANGE') {
|
|
||||||
// throw new Error(`[TabsController]: Invalid message type: ${message.type}`)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
if (message.action === 'created') {
|
|
||||||
const tab = message.payload.tab as chrome.tabs.Tab
|
|
||||||
if (tab.groupId === this.tabGroupId && tab.id != null) {
|
|
||||||
// Tab created in our controlled group
|
|
||||||
if (!this.tabs.find((t) => t.id === tab.id)) {
|
|
||||||
this.tabs.push({ id: tab.id, isInitial: false })
|
|
||||||
}
|
|
||||||
this.switchToTab(tab.id)
|
|
||||||
}
|
|
||||||
} else if (message.action === 'removed') {
|
|
||||||
const { tabId } = message.payload as { tabId: number }
|
|
||||||
const targetTab = this.tabs.find((t) => t.id === tabId)
|
|
||||||
if (targetTab) {
|
|
||||||
this.tabs = this.tabs.filter((t) => t.id !== tabId)
|
|
||||||
if (this.currentTabId === tabId) {
|
|
||||||
const newCurrentTab = this.tabs[this.tabs.length - 1] || null
|
|
||||||
if (newCurrentTab) {
|
|
||||||
this.switchToTab(newCurrentTab.id)
|
|
||||||
} else {
|
|
||||||
this.updateCurrentTabId(null)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
} else if (message.action === 'updated') {
|
|
||||||
const { tabId, tab } = message.payload as { tabId: number; tab: chrome.tabs.Tab }
|
|
||||||
const targetTab = this.tabs.find((t) => t.id === tabId)
|
|
||||||
if (targetTab) {
|
|
||||||
targetTab.url = tab.url
|
|
||||||
targetTab.title = tab.title
|
|
||||||
targetTab.status = tab.status
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
chrome.runtime.onMessage.addListener(tabChangeHandler)
|
|
||||||
|
|
||||||
this.addEventListener('dispose', () => {
|
|
||||||
chrome.runtime.onMessage.removeListener(tabChangeHandler)
|
|
||||||
})
|
|
||||||
}
|
}
|
||||||
|
|
||||||
async openNewTab(url: string): Promise<string> {
|
async openNewTab(url: string): Promise<string> {
|
||||||
@@ -136,7 +133,7 @@ export class TabsController extends EventTarget {
|
|||||||
|
|
||||||
const tabId = result.tabId as number
|
const tabId = result.tabId as number
|
||||||
|
|
||||||
this.tabs.push({
|
this.addTab({
|
||||||
id: tabId,
|
id: tabId,
|
||||||
isInitial: false,
|
isInitial: false,
|
||||||
})
|
})
|
||||||
@@ -209,7 +206,7 @@ export class TabsController extends EventTarget {
|
|||||||
const result = await sendMessage({
|
const result = await sendMessage({
|
||||||
type: 'TAB_CONTROL',
|
type: 'TAB_CONTROL',
|
||||||
action: 'create_tab_group',
|
action: 'create_tab_group',
|
||||||
payload: { tabIds },
|
payload: { tabIds, windowId: this.windowId },
|
||||||
})
|
})
|
||||||
|
|
||||||
if (!result?.success) {
|
if (!result?.success) {
|
||||||
@@ -232,6 +229,11 @@ export class TabsController extends EventTarget {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private addTab(meta: TabMeta) {
|
||||||
|
if (this.tabs.find((t) => t.id === meta.id)) return
|
||||||
|
this.tabs.push(meta)
|
||||||
|
}
|
||||||
|
|
||||||
async updateCurrentTabId(tabId: number | null) {
|
async updateCurrentTabId(tabId: number | null) {
|
||||||
debug('updateCurrentTabId', tabId)
|
debug('updateCurrentTabId', tabId)
|
||||||
|
|
||||||
@@ -288,9 +290,77 @@ export class TabsController extends EventTarget {
|
|||||||
await waitUntil(() => tab.status === 'complete', 4_000)
|
await waitUntil(() => tab.status === 'complete', 4_000)
|
||||||
}
|
}
|
||||||
|
|
||||||
dispose() {
|
/**
|
||||||
this.dispatchEvent(new Event('dispose'))
|
* Connect to background SW via port to receive tab change events.
|
||||||
|
*
|
||||||
|
* @note Port is 1:1 (runtime.connect → background SW has no frames),
|
||||||
|
* so onDisconnect fires exactly once and we can safely reconnect.
|
||||||
|
* Reconnection may miss events during the gap.
|
||||||
|
* TODO: refresh this.tabs from background after reconnect to stay consistent.
|
||||||
|
*/
|
||||||
|
private connectTabEvents() {
|
||||||
|
this.port = chrome.runtime.connect({ name: 'tab-events' })
|
||||||
|
|
||||||
|
this.port.onMessage.addListener((message: any) => {
|
||||||
|
if (this.disposed) return
|
||||||
|
this.portRetries = 0
|
||||||
|
|
||||||
|
if (message.action === 'created') {
|
||||||
|
const tab = message.payload.tab as chrome.tabs.Tab
|
||||||
|
const shouldTrack = this.experimentalIncludeAllTabs || tab.groupId === this.tabGroupId
|
||||||
|
if (shouldTrack && tab.id != null) {
|
||||||
|
this.addTab({ id: tab.id, isInitial: false })
|
||||||
|
this.switchToTab(tab.id)
|
||||||
|
}
|
||||||
|
} else if (message.action === 'removed') {
|
||||||
|
const { tabId } = message.payload as { tabId: number }
|
||||||
|
const targetTab = this.tabs.find((t) => t.id === tabId)
|
||||||
|
if (targetTab) {
|
||||||
|
this.tabs = this.tabs.filter((t) => t.id !== tabId)
|
||||||
|
if (this.currentTabId === tabId) {
|
||||||
|
const newCurrentTab = this.tabs[this.tabs.length - 1] || null
|
||||||
|
if (newCurrentTab) {
|
||||||
|
this.switchToTab(newCurrentTab.id)
|
||||||
|
} else {
|
||||||
|
this.updateCurrentTabId(null)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else if (message.action === 'updated') {
|
||||||
|
const { tabId, tab } = message.payload as { tabId: number; tab: chrome.tabs.Tab }
|
||||||
|
const targetTab = this.tabs.find((t) => t.id === tabId)
|
||||||
|
if (targetTab) {
|
||||||
|
targetTab.url = tab.url
|
||||||
|
targetTab.title = tab.title
|
||||||
|
targetTab.status = tab.status
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
this.port.onDisconnect.addListener(() => {
|
||||||
|
this.port = null
|
||||||
|
if (this.disposed) return
|
||||||
|
if (this.portRetries >= 7) {
|
||||||
|
console.error(PREFIX, 'tab events port failed after 7 retries, giving up')
|
||||||
|
return
|
||||||
|
}
|
||||||
|
debug('port disconnected, reconnecting...')
|
||||||
|
this.portRetries++
|
||||||
|
this.connectTabEvents()
|
||||||
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
dispose() {
|
||||||
|
debug('dispose')
|
||||||
|
this.disposed = true
|
||||||
|
this.port?.disconnect()
|
||||||
|
this.port = null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface TabsInitOptions {
|
||||||
|
includeInitialTab?: boolean
|
||||||
|
experimentalIncludeAllTabs?: boolean
|
||||||
}
|
}
|
||||||
|
|
||||||
export type TabAction =
|
export type TabAction =
|
||||||
@@ -302,6 +372,7 @@ export type TabAction =
|
|||||||
| 'add_tab_to_group'
|
| 'add_tab_to_group'
|
||||||
| 'close_tab'
|
| 'close_tab'
|
||||||
| 'get_tab_title'
|
| 'get_tab_title'
|
||||||
|
| 'get_window_tabs'
|
||||||
|
|
||||||
interface TabMeta {
|
interface TabMeta {
|
||||||
id: number
|
id: number
|
||||||
|
|||||||
@@ -21,6 +21,7 @@ export interface AdvancedConfig {
|
|||||||
maxSteps?: number
|
maxSteps?: number
|
||||||
systemInstruction?: string
|
systemInstruction?: string
|
||||||
experimentalLlmsTxt?: boolean
|
experimentalLlmsTxt?: boolean
|
||||||
|
experimentalIncludeAllTabs?: boolean
|
||||||
disableNamedToolChoice?: boolean
|
disableNamedToolChoice?: boolean
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -125,6 +126,7 @@ export function useAgent(): UseAgentResult {
|
|||||||
maxSteps,
|
maxSteps,
|
||||||
systemInstruction,
|
systemInstruction,
|
||||||
experimentalLlmsTxt,
|
experimentalLlmsTxt,
|
||||||
|
experimentalIncludeAllTabs,
|
||||||
disableNamedToolChoice,
|
disableNamedToolChoice,
|
||||||
...llmConfig
|
...llmConfig
|
||||||
}: ExtConfig) => {
|
}: ExtConfig) => {
|
||||||
@@ -138,6 +140,7 @@ export function useAgent(): UseAgentResult {
|
|||||||
maxSteps,
|
maxSteps,
|
||||||
systemInstruction,
|
systemInstruction,
|
||||||
experimentalLlmsTxt,
|
experimentalLlmsTxt,
|
||||||
|
experimentalIncludeAllTabs,
|
||||||
disableNamedToolChoice,
|
disableNamedToolChoice,
|
||||||
}
|
}
|
||||||
await chrome.storage.local.set({ advancedConfig })
|
await chrome.storage.local.set({ advancedConfig })
|
||||||
|
|||||||
@@ -31,17 +31,20 @@ export function ConfigPanel({ config, onSave, onClose }: ConfigPanelProps) {
|
|||||||
const [model, setModel] = useState(config?.model || DEMO_MODEL)
|
const [model, setModel] = useState(config?.model || DEMO_MODEL)
|
||||||
const [apiKey, setApiKey] = useState(config?.apiKey)
|
const [apiKey, setApiKey] = useState(config?.apiKey)
|
||||||
const [language, setLanguage] = useState<LanguagePreference>(config?.language)
|
const [language, setLanguage] = useState<LanguagePreference>(config?.language)
|
||||||
const [maxSteps, setMaxSteps] = useState<number | undefined>(config?.maxSteps)
|
const [maxSteps, setMaxSteps] = useState(config?.maxSteps)
|
||||||
const [systemInstruction, setSystemInstruction] = useState(config?.systemInstruction ?? '')
|
const [systemInstruction, setSystemInstruction] = useState(config?.systemInstruction ?? '')
|
||||||
const [experimentalLlmsTxt, setExperimentalLlmsTxt] = useState(
|
const [experimentalLlmsTxt, setExperimentalLlmsTxt] = useState(
|
||||||
config?.experimentalLlmsTxt ?? false
|
config?.experimentalLlmsTxt ?? false
|
||||||
)
|
)
|
||||||
|
const [experimentalIncludeAllTabs, setExperimentalIncludeAllTabs] = useState(
|
||||||
|
config?.experimentalIncludeAllTabs ?? false
|
||||||
|
)
|
||||||
const [disableNamedToolChoice, setDisableNamedToolChoice] = useState(
|
const [disableNamedToolChoice, setDisableNamedToolChoice] = useState(
|
||||||
config?.disableNamedToolChoice ?? false
|
config?.disableNamedToolChoice ?? false
|
||||||
)
|
)
|
||||||
const [advancedOpen, setAdvancedOpen] = useState(false)
|
const [advancedOpen, setAdvancedOpen] = useState(false)
|
||||||
const [saving, setSaving] = useState(false)
|
const [saving, setSaving] = useState(false)
|
||||||
const [userAuthToken, setUserAuthToken] = useState<string>('')
|
const [userAuthToken, setUserAuthToken] = useState('')
|
||||||
const [copied, setCopied] = useState(false)
|
const [copied, setCopied] = useState(false)
|
||||||
const [showToken, setShowToken] = useState(false)
|
const [showToken, setShowToken] = useState(false)
|
||||||
const [showApiKey, setShowApiKey] = useState(false)
|
const [showApiKey, setShowApiKey] = useState(false)
|
||||||
@@ -54,6 +57,7 @@ export function ConfigPanel({ config, onSave, onClose }: ConfigPanelProps) {
|
|||||||
setMaxSteps(config?.maxSteps)
|
setMaxSteps(config?.maxSteps)
|
||||||
setSystemInstruction(config?.systemInstruction ?? '')
|
setSystemInstruction(config?.systemInstruction ?? '')
|
||||||
setExperimentalLlmsTxt(config?.experimentalLlmsTxt ?? false)
|
setExperimentalLlmsTxt(config?.experimentalLlmsTxt ?? false)
|
||||||
|
setExperimentalIncludeAllTabs(config?.experimentalIncludeAllTabs ?? false)
|
||||||
setDisableNamedToolChoice(config?.disableNamedToolChoice ?? false)
|
setDisableNamedToolChoice(config?.disableNamedToolChoice ?? false)
|
||||||
}, [config])
|
}, [config])
|
||||||
|
|
||||||
@@ -100,6 +104,7 @@ export function ConfigPanel({ config, onSave, onClose }: ConfigPanelProps) {
|
|||||||
maxSteps: maxSteps || undefined,
|
maxSteps: maxSteps || undefined,
|
||||||
systemInstruction: systemInstruction || undefined,
|
systemInstruction: systemInstruction || undefined,
|
||||||
experimentalLlmsTxt,
|
experimentalLlmsTxt,
|
||||||
|
experimentalIncludeAllTabs,
|
||||||
disableNamedToolChoice,
|
disableNamedToolChoice,
|
||||||
})
|
})
|
||||||
} finally {
|
} finally {
|
||||||
@@ -285,6 +290,14 @@ export function ConfigPanel({ config, onSave, onClose }: ConfigPanelProps) {
|
|||||||
<span className="text-xs text-muted-foreground">Experimental llms.txt support</span>
|
<span className="text-xs text-muted-foreground">Experimental llms.txt support</span>
|
||||||
<Switch checked={experimentalLlmsTxt} onCheckedChange={setExperimentalLlmsTxt} />
|
<Switch checked={experimentalLlmsTxt} onCheckedChange={setExperimentalLlmsTxt} />
|
||||||
</label>
|
</label>
|
||||||
|
|
||||||
|
<label className="flex items-center justify-between cursor-pointer">
|
||||||
|
<span className="text-xs text-muted-foreground">Experimental include all tabs</span>
|
||||||
|
<Switch
|
||||||
|
checked={experimentalIncludeAllTabs}
|
||||||
|
onCheckedChange={setExperimentalIncludeAllTabs}
|
||||||
|
/>
|
||||||
|
</label>
|
||||||
</>
|
</>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
|
|||||||
@@ -111,6 +111,7 @@ export function EmptyState() {
|
|||||||
]}
|
]}
|
||||||
cursorStyle="underscore"
|
cursorStyle="underscore"
|
||||||
loop
|
loop
|
||||||
|
startOnView={false}
|
||||||
typeSpeed={20}
|
typeSpeed={20}
|
||||||
deleteSpeed={10}
|
deleteSpeed={10}
|
||||||
pauseDelay={3000}
|
pauseDelay={3000}
|
||||||
|
|||||||
@@ -1,12 +1,12 @@
|
|||||||
import { handlePageControlMessage } from '@/agent/RemotePageController.background'
|
import { handlePageControlMessage } from '@/agent/RemotePageController.background'
|
||||||
import { handleTabControlMessage, setupTabChangeEvents } from '@/agent/TabsController.background'
|
import { handleTabControlMessage, setupTabEventsPort } from '@/agent/TabsController.background'
|
||||||
|
|
||||||
export default defineBackground(() => {
|
export default defineBackground(() => {
|
||||||
console.log('[Background] Service Worker started')
|
console.log('[Background] Service Worker started')
|
||||||
|
|
||||||
// tab change events
|
// tab change events
|
||||||
|
|
||||||
setupTabChangeEvents()
|
setupTabEventsPort()
|
||||||
|
|
||||||
// generate user auth token
|
// generate user auth token
|
||||||
|
|
||||||
|
|||||||
@@ -46,6 +46,8 @@ async function exposeAgentToPage() {
|
|||||||
let multiPageAgent: InstanceType<typeof MultiPageAgent> | null = null
|
let multiPageAgent: InstanceType<typeof MultiPageAgent> | null = null
|
||||||
|
|
||||||
window.addEventListener('message', async (e) => {
|
window.addEventListener('message', async (e) => {
|
||||||
|
if (e.source !== window) return
|
||||||
|
|
||||||
const data = e.data
|
const data = e.data
|
||||||
if (typeof data !== 'object' || data === null) return
|
if (typeof data !== 'object' || data === null) return
|
||||||
if (data.channel !== 'PAGE_AGENT_EXT_REQUEST') return
|
if (data.channel !== 'PAGE_AGENT_EXT_REQUEST') return
|
||||||
@@ -70,11 +72,15 @@ async function exposeAgentToPage() {
|
|||||||
|
|
||||||
try {
|
try {
|
||||||
const { task, config } = payload
|
const { task, config } = payload
|
||||||
|
const { systemInstruction, ...agentConfig } = config
|
||||||
|
|
||||||
// Dispose old instance before creating new one
|
// Dispose old instance before creating new one
|
||||||
multiPageAgent?.dispose()
|
multiPageAgent?.dispose()
|
||||||
|
|
||||||
multiPageAgent = new MultiPageAgent(config)
|
multiPageAgent = new MultiPageAgent({
|
||||||
|
...agentConfig,
|
||||||
|
instructions: systemInstruction ? { system: systemInstruction } : undefined,
|
||||||
|
})
|
||||||
|
|
||||||
// events
|
// events
|
||||||
|
|
||||||
|
|||||||
@@ -7,12 +7,21 @@ export interface ExecuteConfig {
|
|||||||
model: string
|
model: string
|
||||||
apiKey?: string
|
apiKey?: string
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Global system-level instructions for the agent.
|
||||||
|
* Equivalent to `AgentConfig.instructions.system`.
|
||||||
|
*/
|
||||||
|
systemInstruction?: string
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Whether to include the initial tab (that holds this main world script) in the task.
|
* Whether to include the initial tab (that holds this main world script) in the task.
|
||||||
* @default true
|
* @default true
|
||||||
*/
|
*/
|
||||||
includeInitialTab?: boolean
|
includeInitialTab?: boolean
|
||||||
|
|
||||||
|
/** Control all unpinned tabs in the window instead of only the tab group. */
|
||||||
|
experimentalIncludeAllTabs?: boolean
|
||||||
|
|
||||||
onStatusChange?: (status: AgentStatus) => void
|
onStatusChange?: (status: AgentStatus) => void
|
||||||
onActivity?: (activity: AgentActivity) => void
|
onActivity?: (activity: AgentActivity) => void
|
||||||
onHistoryUpdate?: (history: HistoricalEvent[]) => void
|
onHistoryUpdate?: (history: HistoricalEvent[]) => void
|
||||||
@@ -36,6 +45,8 @@ export default defineUnlistedScript(() => {
|
|||||||
|
|
||||||
const promise = new Promise<ExecutionResult>((resolve, reject) => {
|
const promise = new Promise<ExecutionResult>((resolve, reject) => {
|
||||||
function handleMessage(e: MessageEvent) {
|
function handleMessage(e: MessageEvent) {
|
||||||
|
if (e.source !== window) return
|
||||||
|
|
||||||
const data = e.data
|
const data = e.data
|
||||||
if (typeof data !== 'object' || data === null) return
|
if (typeof data !== 'object' || data === null) return
|
||||||
if (data.channel !== 'PAGE_AGENT_EXT_RESPONSE') return
|
if (data.channel !== 'PAGE_AGENT_EXT_RESPONSE') return
|
||||||
@@ -86,7 +97,9 @@ export default defineUnlistedScript(() => {
|
|||||||
baseURL: config.baseURL,
|
baseURL: config.baseURL,
|
||||||
model: config.model,
|
model: config.model,
|
||||||
apiKey: config.apiKey,
|
apiKey: config.apiKey,
|
||||||
|
systemInstruction: config.systemInstruction,
|
||||||
includeInitialTab: config.includeInitialTab,
|
includeInitialTab: config.includeInitialTab,
|
||||||
|
experimentalIncludeAllTabs: config.experimentalIncludeAllTabs,
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@page-agent/llms",
|
"name": "@page-agent/llms",
|
||||||
"version": "1.6.1",
|
"version": "1.7.1",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "./dist/lib/page-agent-llms.js",
|
"main": "./dist/lib/page-agent-llms.js",
|
||||||
"module": "./dist/lib/page-agent-llms.js",
|
"module": "./dist/lib/page-agent-llms.js",
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
{
|
{
|
||||||
"name": "@page-agent/mcp",
|
"name": "@page-agent/mcp",
|
||||||
"private": false,
|
"private": false,
|
||||||
"version": "1.6.1",
|
"version": "1.7.1",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"bin": {
|
"bin": {
|
||||||
"page-agent-mcp": "src/index.js"
|
"page-agent-mcp": "src/index.js"
|
||||||
@@ -28,8 +28,8 @@
|
|||||||
"node": ">=20"
|
"node": ">=20"
|
||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@modelcontextprotocol/sdk": "^1.27.1",
|
"@modelcontextprotocol/sdk": "^1.29.0",
|
||||||
"ws": "^8.19.0",
|
"ws": "^8.20.0",
|
||||||
"zod": "^4.3.5"
|
"zod": "^4.3.5"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
{
|
{
|
||||||
"name": "page-agent",
|
"name": "page-agent",
|
||||||
"private": false,
|
"private": false,
|
||||||
"version": "1.6.1",
|
"version": "1.7.1",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "./dist/esm/page-agent.js",
|
"main": "./dist/esm/page-agent.js",
|
||||||
"module": "./dist/esm/page-agent.js",
|
"module": "./dist/esm/page-agent.js",
|
||||||
@@ -44,10 +44,10 @@
|
|||||||
"postpublish": "node -e \"['README.md','LICENSE'].forEach(f=>{try{require('fs').unlinkSync(f)}catch{}})\""
|
"postpublish": "node -e \"['README.md','LICENSE'].forEach(f=>{try{require('fs').unlinkSync(f)}catch{}})\""
|
||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@page-agent/core": "1.6.1",
|
"@page-agent/core": "1.7.1",
|
||||||
"@page-agent/llms": "1.6.1",
|
"@page-agent/llms": "1.7.1",
|
||||||
"@page-agent/page-controller": "1.6.1",
|
"@page-agent/page-controller": "1.7.1",
|
||||||
"@page-agent/ui": "1.6.1",
|
"@page-agent/ui": "1.7.1",
|
||||||
"chalk": "^5.6.2"
|
"chalk": "^5.6.2"
|
||||||
},
|
},
|
||||||
"peerDependencies": {
|
"peerDependencies": {
|
||||||
|
|||||||
@@ -4,11 +4,11 @@
|
|||||||
*/
|
*/
|
||||||
import { type AgentConfig, PageAgentCore } from '@page-agent/core'
|
import { type AgentConfig, PageAgentCore } from '@page-agent/core'
|
||||||
import { PageController, type PageControllerConfig } from '@page-agent/page-controller'
|
import { PageController, type PageControllerConfig } from '@page-agent/page-controller'
|
||||||
import { Panel } from '@page-agent/ui'
|
import { Panel, type PanelConfig } from '@page-agent/ui'
|
||||||
|
|
||||||
export * from '@page-agent/core'
|
export * from '@page-agent/core'
|
||||||
|
|
||||||
export type PageAgentConfig = AgentConfig & PageControllerConfig
|
export type PageAgentConfig = AgentConfig & PageControllerConfig & Omit<PanelConfig, 'language'>
|
||||||
|
|
||||||
export class PageAgent extends PageAgentCore {
|
export class PageAgent extends PageAgentCore {
|
||||||
panel: Panel
|
panel: Panel
|
||||||
@@ -23,6 +23,7 @@ export class PageAgent extends PageAgentCore {
|
|||||||
|
|
||||||
this.panel = new Panel(this, {
|
this.panel = new Panel(this, {
|
||||||
language: config.language,
|
language: config.language,
|
||||||
|
promptForNextTask: config.promptForNextTask,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -17,9 +17,10 @@ const DEMO_MODEL = 'qwen3.5-plus'
|
|||||||
const DEMO_BASE_URL = 'https://page-ag-testing-ohftxirgbn.cn-shanghai.fcapp.run'
|
const DEMO_BASE_URL = 'https://page-ag-testing-ohftxirgbn.cn-shanghai.fcapp.run'
|
||||||
const DEMO_API_KEY = 'NA'
|
const DEMO_API_KEY = 'NA'
|
||||||
|
|
||||||
|
const currentScript = document.currentScript as HTMLScriptElement | null
|
||||||
|
|
||||||
// in case document.x is not ready yet
|
// in case document.x is not ready yet
|
||||||
setTimeout(() => {
|
setTimeout(() => {
|
||||||
const currentScript = document.currentScript as HTMLScriptElement | null
|
|
||||||
let config: PageAgentConfig
|
let config: PageAgentConfig
|
||||||
|
|
||||||
if (currentScript) {
|
if (currentScript) {
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@page-agent/page-controller",
|
"name": "@page-agent/page-controller",
|
||||||
"version": "1.6.1",
|
"version": "1.7.1",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "./dist/lib/page-controller.js",
|
"main": "./dist/lib/page-controller.js",
|
||||||
"module": "./dist/lib/page-controller.js",
|
"module": "./dist/lib/page-controller.js",
|
||||||
|
|||||||
@@ -193,7 +193,11 @@ export class PageController extends EventTarget {
|
|||||||
interactiveBlacklist: blacklist,
|
interactiveBlacklist: blacklist,
|
||||||
})
|
})
|
||||||
|
|
||||||
this.simplifiedHTML = dom.flatTreeToString(this.flatTree, this.config.includeAttributes)
|
this.simplifiedHTML = dom.flatTreeToString(
|
||||||
|
this.flatTree,
|
||||||
|
this.config.includeAttributes,
|
||||||
|
this.config.keepSemanticTags
|
||||||
|
)
|
||||||
|
|
||||||
this.selectorMap.clear()
|
this.selectorMap.clear()
|
||||||
this.selectorMap = dom.getSelectorMap(this.flatTree)
|
this.selectorMap = dom.getSelectorMap(this.flatTree)
|
||||||
@@ -218,6 +222,7 @@ export class PageController extends EventTarget {
|
|||||||
* Clean up all element highlights
|
* Clean up all element highlights
|
||||||
*/
|
*/
|
||||||
async cleanUpHighlights(): Promise<void> {
|
async cleanUpHighlights(): Promise<void> {
|
||||||
|
console.log('[PageController] cleanUpHighlights')
|
||||||
dom.cleanUpHighlights()
|
dom.cleanUpHighlights()
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -321,11 +326,11 @@ export class PageController extends EventTarget {
|
|||||||
|
|
||||||
this.assertIndexed()
|
this.assertIndexed()
|
||||||
|
|
||||||
const scrollAmount = pixels ?? numPages * (down ? 1 : -1) * window.innerHeight
|
const scrollAmount = (pixels ?? numPages * window.innerHeight) * (down ? 1 : -1)
|
||||||
|
|
||||||
const element = index !== undefined ? getElementByIndex(this.selectorMap, index) : null
|
const element = index !== undefined ? getElementByIndex(this.selectorMap, index) : null
|
||||||
|
|
||||||
const message = await scrollVertically(down, scrollAmount, element)
|
const message = await scrollVertically(scrollAmount, element)
|
||||||
|
|
||||||
return {
|
return {
|
||||||
success: true,
|
success: true,
|
||||||
@@ -356,7 +361,7 @@ export class PageController extends EventTarget {
|
|||||||
|
|
||||||
const element = index !== undefined ? getElementByIndex(this.selectorMap, index) : null
|
const element = index !== undefined ? getElementByIndex(this.selectorMap, index) : null
|
||||||
|
|
||||||
const message = await scrollHorizontally(right, scrollAmount, element)
|
const message = await scrollHorizontally(scrollAmount, element)
|
||||||
|
|
||||||
return {
|
return {
|
||||||
success: true,
|
success: true,
|
||||||
|
|||||||
@@ -4,6 +4,9 @@
|
|||||||
*/
|
*/
|
||||||
import type { InteractiveElementDomNode } from './dom/dom_tree/type'
|
import type { InteractiveElementDomNode } from './dom/dom_tree/type'
|
||||||
import {
|
import {
|
||||||
|
clickPointer,
|
||||||
|
disablePassThrough,
|
||||||
|
enablePassThrough,
|
||||||
getNativeValueSetter,
|
getNativeValueSetter,
|
||||||
isHTMLElement,
|
isHTMLElement,
|
||||||
isInputElement,
|
isInputElement,
|
||||||
@@ -42,19 +45,20 @@ let lastClickedElement: HTMLElement | null = null
|
|||||||
|
|
||||||
function blurLastClickedElement() {
|
function blurLastClickedElement() {
|
||||||
if (lastClickedElement) {
|
if (lastClickedElement) {
|
||||||
|
lastClickedElement.dispatchEvent(new PointerEvent('pointerout', { bubbles: true }))
|
||||||
|
lastClickedElement.dispatchEvent(new PointerEvent('pointerleave', { bubbles: false }))
|
||||||
|
lastClickedElement.dispatchEvent(new MouseEvent('mouseout', { bubbles: true }))
|
||||||
|
lastClickedElement.dispatchEvent(new MouseEvent('mouseleave', { bubbles: false }))
|
||||||
lastClickedElement.blur()
|
lastClickedElement.blur()
|
||||||
lastClickedElement.dispatchEvent(
|
|
||||||
new MouseEvent('mouseout', { bubbles: true, cancelable: true })
|
|
||||||
)
|
|
||||||
lastClickedElement.dispatchEvent(
|
|
||||||
new MouseEvent('mouseleave', { bubbles: false, cancelable: true })
|
|
||||||
)
|
|
||||||
lastClickedElement = null
|
lastClickedElement = null
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Simulate a click on the element
|
* Simulate a full click following W3C Pointer Events + UI Events spec order:
|
||||||
|
* pointerover/enter → mouseover/enter → pointerdown → mousedown → [focus] →
|
||||||
|
* pointerup → mouseup → click
|
||||||
|
*
|
||||||
* @private Internal method, subject to change at any time.
|
* @private Internal method, subject to change at any time.
|
||||||
*/
|
*/
|
||||||
export async function clickElement(element: HTMLElement) {
|
export async function clickElement(element: HTMLElement) {
|
||||||
@@ -63,32 +67,62 @@ export async function clickElement(element: HTMLElement) {
|
|||||||
lastClickedElement = element
|
lastClickedElement = element
|
||||||
|
|
||||||
await scrollIntoViewIfNeeded(element)
|
await scrollIntoViewIfNeeded(element)
|
||||||
// Scroll the iframe element itself into view if needed
|
|
||||||
const frame = element.ownerDocument.defaultView?.frameElement
|
const frame = element.ownerDocument.defaultView?.frameElement
|
||||||
if (frame) await scrollIntoViewIfNeeded(frame)
|
if (frame) await scrollIntoViewIfNeeded(frame)
|
||||||
|
|
||||||
await movePointerToElement(element)
|
const rect = element.getBoundingClientRect()
|
||||||
window.dispatchEvent(new CustomEvent('PageAgent::ClickPointer'))
|
const x = rect.left + rect.width / 2
|
||||||
|
const y = rect.top + rect.height / 2
|
||||||
|
|
||||||
|
await movePointerToElement(element, x, y)
|
||||||
|
await clickPointer()
|
||||||
|
|
||||||
await waitFor(0.1)
|
await waitFor(0.1)
|
||||||
|
|
||||||
// hover it
|
// Hit-test to find the deepest element at click coordinates, matching
|
||||||
element.dispatchEvent(new MouseEvent('mouseenter', { bubbles: true, cancelable: true }))
|
// real browser behavior where events target the innermost element.
|
||||||
element.dispatchEvent(new MouseEvent('mouseover', { bubbles: true, cancelable: true }))
|
// @note This may hit a element in the blacklist
|
||||||
|
// TODO: This is a temporary workaround. Should have been handled during dom extraction.
|
||||||
|
const doc = element.ownerDocument
|
||||||
|
await enablePassThrough()
|
||||||
|
const hitTarget = doc.elementFromPoint(x, y)
|
||||||
|
await disablePassThrough()
|
||||||
|
const target =
|
||||||
|
hitTarget instanceof HTMLElement && element.contains(hitTarget) ? hitTarget : element
|
||||||
|
|
||||||
// dispatch a sequence of events to ensure all listeners are triggered
|
const pointerOpts = {
|
||||||
element.dispatchEvent(new MouseEvent('mousedown', { bubbles: true, cancelable: true }))
|
bubbles: true,
|
||||||
|
cancelable: true,
|
||||||
|
clientX: x,
|
||||||
|
clientY: y,
|
||||||
|
pointerType: 'mouse',
|
||||||
|
}
|
||||||
|
const mouseOpts = { bubbles: true, cancelable: true, clientX: x, clientY: y, button: 0 }
|
||||||
|
|
||||||
// focus it to ensure it gets the click event
|
// Hover — pointer events first, then mouse events (spec order)
|
||||||
element.focus()
|
target.dispatchEvent(new PointerEvent('pointerover', pointerOpts))
|
||||||
|
target.dispatchEvent(new PointerEvent('pointerenter', { ...pointerOpts, bubbles: false }))
|
||||||
|
target.dispatchEvent(new MouseEvent('mouseover', mouseOpts))
|
||||||
|
target.dispatchEvent(new MouseEvent('mouseenter', { ...mouseOpts, bubbles: false }))
|
||||||
|
|
||||||
element.dispatchEvent(new MouseEvent('mouseup', { bubbles: true, cancelable: true }))
|
// Press
|
||||||
element.dispatchEvent(new MouseEvent('click', { bubbles: true, cancelable: true }))
|
target.dispatchEvent(new PointerEvent('pointerdown', pointerOpts))
|
||||||
|
target.dispatchEvent(new MouseEvent('mousedown', mouseOpts))
|
||||||
|
|
||||||
// dispatch a click event
|
// Focus is not part of the standard pointer/mouse event sequence
|
||||||
// element.click()
|
// "undefined and varies between user agents".
|
||||||
|
// We focus the original element (nearest focusable ancestor), not the hit-test target, matching browser behavior.
|
||||||
|
element.focus({ preventScroll: true })
|
||||||
|
|
||||||
await waitFor(0.2) // Wait to ensure click event processing completes
|
// Release
|
||||||
|
target.dispatchEvent(new PointerEvent('pointerup', pointerOpts))
|
||||||
|
target.dispatchEvent(new MouseEvent('mouseup', mouseOpts))
|
||||||
|
|
||||||
|
// Click — activation behavior (navigation, form submit, etc.) triggers
|
||||||
|
// via bubbling from target up to the interactive ancestor.
|
||||||
|
target.click()
|
||||||
|
|
||||||
|
await waitFor(0.2)
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -240,14 +274,7 @@ export async function scrollIntoViewIfNeeded(element: Element) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
export async function scrollVertically(scroll_amount: number, element?: HTMLElement | null) {
|
||||||
* @private Internal method, subject to change at any time.
|
|
||||||
*/
|
|
||||||
export async function scrollVertically(
|
|
||||||
down: boolean,
|
|
||||||
scroll_amount: number,
|
|
||||||
element?: HTMLElement | null
|
|
||||||
) {
|
|
||||||
// Element-specific scrolling if element is provided
|
// Element-specific scrolling if element is provided
|
||||||
if (element) {
|
if (element) {
|
||||||
const targetElement = element
|
const targetElement = element
|
||||||
@@ -260,7 +287,10 @@ export async function scrollVertically(
|
|||||||
|
|
||||||
while (currentElement && attempts < 10) {
|
while (currentElement && attempts < 10) {
|
||||||
const computedStyle = window.getComputedStyle(currentElement)
|
const computedStyle = window.getComputedStyle(currentElement)
|
||||||
const hasScrollableY = /(auto|scroll|overlay)/.test(computedStyle.overflowY)
|
const hasScrollableY =
|
||||||
|
/(auto|scroll|overlay)/.test(computedStyle.overflowY) ||
|
||||||
|
(computedStyle.scrollbarWidth && computedStyle.scrollbarWidth !== 'auto') ||
|
||||||
|
(computedStyle.scrollbarGutter && computedStyle.scrollbarGutter !== 'auto')
|
||||||
const canScrollVertically = currentElement.scrollHeight > currentElement.clientHeight
|
const canScrollVertically = currentElement.scrollHeight > currentElement.clientHeight
|
||||||
|
|
||||||
if (hasScrollableY && canScrollVertically) {
|
if (hasScrollableY && canScrollVertically) {
|
||||||
@@ -312,9 +342,20 @@ export async function scrollVertically(
|
|||||||
el.scrollHeight > el.clientHeight &&
|
el.scrollHeight > el.clientHeight &&
|
||||||
bigEnough(el)
|
bigEnough(el)
|
||||||
|
|
||||||
|
// @deprecated Heuristic container search.
|
||||||
|
// Unreliable in multi-panel layouts. Should guide LLMs to use indexed scroll for consistency.
|
||||||
|
// TODO: remove this fallback
|
||||||
|
|
||||||
|
// try to find the nearest scrollable container
|
||||||
|
// document.activeElement is usually body.
|
||||||
|
// After a successful element.focus(), activeElement become the nearest focusable parent
|
||||||
|
|
||||||
let el: HTMLElement | null = document.activeElement as HTMLElement | null
|
let el: HTMLElement | null = document.activeElement as HTMLElement | null
|
||||||
while (el && !canScroll(el) && el !== document.body) el = el.parentElement
|
while (el && !canScroll(el) && el !== document.body) el = el.parentElement
|
||||||
|
|
||||||
|
// Something is wrong if it falls back to global '*' search
|
||||||
|
// TODO: Return error message instead of global '*' search
|
||||||
|
|
||||||
el = canScroll(el)
|
el = canScroll(el)
|
||||||
? el
|
? el
|
||||||
: Array.from(document.querySelectorAll<HTMLElement>('*')).find(canScroll) ||
|
: Array.from(document.querySelectorAll<HTMLElement>('*')).find(canScroll) ||
|
||||||
@@ -345,6 +386,10 @@ export async function scrollVertically(
|
|||||||
return `✅ Scrolled page by ${scrolled}px.`
|
return `✅ Scrolled page by ${scrolled}px.`
|
||||||
} else {
|
} else {
|
||||||
// Container scroll
|
// Container scroll
|
||||||
|
|
||||||
|
const warningMsg = `The document is not scrollable. Falling back to container scroll.`
|
||||||
|
console.log(`[PageController] ${warningMsg}`)
|
||||||
|
|
||||||
const scrollBefore = el!.scrollTop
|
const scrollBefore = el!.scrollTop
|
||||||
const scrollMax = el!.scrollHeight - el!.clientHeight
|
const scrollMax = el!.scrollHeight - el!.clientHeight
|
||||||
|
|
||||||
@@ -356,29 +401,22 @@ export async function scrollVertically(
|
|||||||
|
|
||||||
if (Math.abs(scrolled) < 1) {
|
if (Math.abs(scrolled) < 1) {
|
||||||
return dy > 0
|
return dy > 0
|
||||||
? `⚠️ Already at the bottom of container (${el!.tagName}), cannot scroll down further.`
|
? `⚠️ ${warningMsg} Already at the bottom of container (${el!.tagName}), cannot scroll down further.`
|
||||||
: `⚠️ Already at the top of container (${el!.tagName}), cannot scroll up further.`
|
: `⚠️ ${warningMsg} Already at the top of container (${el!.tagName}), cannot scroll up further.`
|
||||||
}
|
}
|
||||||
|
|
||||||
const reachedBottom = dy > 0 && scrollAfter >= scrollMax - 1
|
const reachedBottom = dy > 0 && scrollAfter >= scrollMax - 1
|
||||||
const reachedTop = dy < 0 && scrollAfter <= 1
|
const reachedTop = dy < 0 && scrollAfter <= 1
|
||||||
|
|
||||||
if (reachedBottom)
|
if (reachedBottom)
|
||||||
return `✅ Scrolled container (${el!.tagName}) by ${scrolled}px. Reached the bottom.`
|
return `✅ ${warningMsg} Scrolled container (${el!.tagName}) by ${scrolled}px. Reached the bottom.`
|
||||||
if (reachedTop)
|
if (reachedTop)
|
||||||
return `✅ Scrolled container (${el!.tagName}) by ${scrolled}px. Reached the top.`
|
return `✅ ${warningMsg} Scrolled container (${el!.tagName}) by ${scrolled}px. Reached the top.`
|
||||||
return `✅ Scrolled container (${el!.tagName}) by ${scrolled}px.`
|
return `✅ ${warningMsg} Scrolled container (${el!.tagName}) by ${scrolled}px.`
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
export async function scrollHorizontally(scroll_amount: number, element?: HTMLElement | null) {
|
||||||
* @private Internal method, subject to change at any time.
|
|
||||||
*/
|
|
||||||
export async function scrollHorizontally(
|
|
||||||
right: boolean,
|
|
||||||
scroll_amount: number,
|
|
||||||
element?: HTMLElement | null
|
|
||||||
) {
|
|
||||||
// Element-specific scrolling if element is provided
|
// Element-specific scrolling if element is provided
|
||||||
if (element) {
|
if (element) {
|
||||||
const targetElement = element
|
const targetElement = element
|
||||||
@@ -387,11 +425,14 @@ export async function scrollHorizontally(
|
|||||||
let scrolledElement: HTMLElement | null = null
|
let scrolledElement: HTMLElement | null = null
|
||||||
let scrollDelta = 0
|
let scrollDelta = 0
|
||||||
let attempts = 0
|
let attempts = 0
|
||||||
const dx = right ? scroll_amount : -scroll_amount
|
const dx = scroll_amount
|
||||||
|
|
||||||
while (currentElement && attempts < 10) {
|
while (currentElement && attempts < 10) {
|
||||||
const computedStyle = window.getComputedStyle(currentElement)
|
const computedStyle = window.getComputedStyle(currentElement)
|
||||||
const hasScrollableX = /(auto|scroll|overlay)/.test(computedStyle.overflowX)
|
const hasScrollableX =
|
||||||
|
/(auto|scroll|overlay)/.test(computedStyle.overflowX) ||
|
||||||
|
(computedStyle.scrollbarWidth && computedStyle.scrollbarWidth !== 'auto') ||
|
||||||
|
(computedStyle.scrollbarGutter && computedStyle.scrollbarGutter !== 'auto')
|
||||||
const canScrollHorizontally = currentElement.scrollWidth > currentElement.clientWidth
|
const canScrollHorizontally = currentElement.scrollWidth > currentElement.clientWidth
|
||||||
|
|
||||||
if (hasScrollableX && canScrollHorizontally) {
|
if (hasScrollableX && canScrollHorizontally) {
|
||||||
@@ -435,7 +476,8 @@ export async function scrollHorizontally(
|
|||||||
|
|
||||||
// Page-level scrolling (default or fallback)
|
// Page-level scrolling (default or fallback)
|
||||||
|
|
||||||
const dx = right ? scroll_amount : -scroll_amount
|
const dx = scroll_amount
|
||||||
|
|
||||||
const bigEnough = (el: HTMLElement) => el.clientWidth >= window.innerWidth * 0.5
|
const bigEnough = (el: HTMLElement) => el.clientWidth >= window.innerWidth * 0.5
|
||||||
const canScroll = (el: HTMLElement | null) =>
|
const canScroll = (el: HTMLElement | null) =>
|
||||||
el &&
|
el &&
|
||||||
@@ -443,6 +485,9 @@ export async function scrollHorizontally(
|
|||||||
el.scrollWidth > el.clientWidth &&
|
el.scrollWidth > el.clientWidth &&
|
||||||
bigEnough(el)
|
bigEnough(el)
|
||||||
|
|
||||||
|
// @deprecated Same heuristic container search as scrollVertically.
|
||||||
|
// TODO: Remove once LLMs reliably use indexed scrolling via data-scrollable.
|
||||||
|
|
||||||
let el: HTMLElement | null = document.activeElement as HTMLElement | null
|
let el: HTMLElement | null = document.activeElement as HTMLElement | null
|
||||||
while (el && !canScroll(el) && el !== document.body) el = el.parentElement
|
while (el && !canScroll(el) && el !== document.body) el = el.parentElement
|
||||||
|
|
||||||
@@ -477,6 +522,9 @@ export async function scrollHorizontally(
|
|||||||
return `✅ Scrolled page horizontally by ${scrolled}px.`
|
return `✅ Scrolled page horizontally by ${scrolled}px.`
|
||||||
} else {
|
} else {
|
||||||
// Container scroll
|
// Container scroll
|
||||||
|
const warningMsg = `The document is not scrollable. Falling back to container scroll.`
|
||||||
|
console.log(`[PageController] ${warningMsg}`)
|
||||||
|
|
||||||
const scrollBefore = el!.scrollLeft
|
const scrollBefore = el!.scrollLeft
|
||||||
const scrollMax = el!.scrollWidth - el!.clientWidth
|
const scrollMax = el!.scrollWidth - el!.clientWidth
|
||||||
|
|
||||||
@@ -488,17 +536,17 @@ export async function scrollHorizontally(
|
|||||||
|
|
||||||
if (Math.abs(scrolled) < 1) {
|
if (Math.abs(scrolled) < 1) {
|
||||||
return dx > 0
|
return dx > 0
|
||||||
? `⚠️ Already at the right edge of container (${el!.tagName}), cannot scroll right further.`
|
? `⚠️ ${warningMsg} Already at the right edge of container (${el!.tagName}), cannot scroll right further.`
|
||||||
: `⚠️ Already at the left edge of container (${el!.tagName}), cannot scroll left further.`
|
: `⚠️ ${warningMsg} Already at the left edge of container (${el!.tagName}), cannot scroll left further.`
|
||||||
}
|
}
|
||||||
|
|
||||||
const reachedRight = dx > 0 && scrollAfter >= scrollMax - 1
|
const reachedRight = dx > 0 && scrollAfter >= scrollMax - 1
|
||||||
const reachedLeft = dx < 0 && scrollAfter <= 1
|
const reachedLeft = dx < 0 && scrollAfter <= 1
|
||||||
|
|
||||||
if (reachedRight)
|
if (reachedRight)
|
||||||
return `✅ Scrolled container (${el!.tagName}) by ${scrolled}px. Reached the right edge.`
|
return `✅ ${warningMsg} Scrolled container (${el!.tagName}) by ${scrolled}px. Reached the right edge.`
|
||||||
if (reachedLeft)
|
if (reachedLeft)
|
||||||
return `✅ Scrolled container (${el!.tagName}) by ${scrolled}px. Reached the left edge.`
|
return `✅ ${warningMsg} Scrolled container (${el!.tagName}) by ${scrolled}px. Reached the left edge.`
|
||||||
return `✅ Scrolled container (${el!.tagName}) horizontally by ${scrolled}px.`
|
return `✅ ${warningMsg} Scrolled container (${el!.tagName}) horizontally by ${scrolled}px.`
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -18,6 +18,7 @@
|
|||||||
* @edit improve `sampleRect`, filter out rects with 0 area
|
* @edit improve `sampleRect`, filter out rects with 0 area
|
||||||
* @edit exclude aria-hidden elements
|
* @edit exclude aria-hidden elements
|
||||||
* @edit make sure attributes exist for interactive candidates.
|
* @edit make sure attributes exist for interactive candidates.
|
||||||
|
* @edit fix "aria-*" attributes check
|
||||||
*/
|
*/
|
||||||
|
|
||||||
export default (
|
export default (
|
||||||
@@ -502,11 +503,16 @@ export default (
|
|||||||
const overflowX = style.overflowX
|
const overflowX = style.overflowX
|
||||||
const overflowY = style.overflowY
|
const overflowY = style.overflowY
|
||||||
|
|
||||||
// Check scrollable distances
|
// scrollbar-width/scrollbar-gutter are only set on elements designed to scroll;
|
||||||
|
// their presence signals scroll intent even when overflow is hidden (e.g. overflow: auto on :hover)
|
||||||
|
const hasScrollbarSignal =
|
||||||
|
(style.scrollbarWidth && style.scrollbarWidth !== 'auto') ||
|
||||||
|
(style.scrollbarGutter && style.scrollbarGutter !== 'auto')
|
||||||
|
|
||||||
const scrollableX = overflowX === 'auto' || overflowX === 'scroll'
|
const scrollableX = overflowX === 'auto' || overflowX === 'scroll'
|
||||||
const scrollableY = overflowY === 'auto' || overflowY === 'scroll'
|
const scrollableY = overflowY === 'auto' || overflowY === 'scroll'
|
||||||
|
|
||||||
if (!scrollableX && !scrollableY) {
|
if (!scrollableX && !scrollableY && !hasScrollbarSignal) {
|
||||||
return null // Not scrollable in any direction
|
return null // Not scrollable in any direction
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -520,11 +526,11 @@ export default (
|
|||||||
return null // Not scrollable
|
return null // Not scrollable
|
||||||
}
|
}
|
||||||
|
|
||||||
if (!scrollableY && scrollWidth < threshold) {
|
if (!scrollableY && !hasScrollbarSignal && scrollWidth < threshold) {
|
||||||
return null // Not scrollable horizontally
|
return null // Not scrollable horizontally
|
||||||
}
|
}
|
||||||
|
|
||||||
if (!scrollableX && scrollHeight < threshold) {
|
if (!scrollableX && !hasScrollbarSignal && scrollHeight < threshold) {
|
||||||
return null // Not scrollable vertically
|
return null // Not scrollable vertically
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -546,6 +552,8 @@ export default (
|
|||||||
scrollData: scrollData,
|
scrollData: scrollData,
|
||||||
})
|
})
|
||||||
|
|
||||||
|
console.log('scrollData!!!', scrollData)
|
||||||
|
|
||||||
return scrollData
|
return scrollData
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1143,6 +1151,31 @@ export default (
|
|||||||
* @param {HTMLElement} element - The element to check.
|
* @param {HTMLElement} element - The element to check.
|
||||||
* @returns {boolean} Whether the element is an interactive candidate.
|
* @returns {boolean} Whether the element is an interactive candidate.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
|
// @edit fix "aria-*" attributes check
|
||||||
|
const INTERACTIVE_ARIA_ATTRS = [
|
||||||
|
'aria-expanded',
|
||||||
|
'aria-checked',
|
||||||
|
'aria-selected',
|
||||||
|
'aria-pressed',
|
||||||
|
'aria-haspopup',
|
||||||
|
'aria-controls',
|
||||||
|
'aria-owns',
|
||||||
|
'aria-activedescendant',
|
||||||
|
'aria-valuenow',
|
||||||
|
'aria-valuetext',
|
||||||
|
'aria-valuemax',
|
||||||
|
'aria-valuemin',
|
||||||
|
'aria-autocomplete',
|
||||||
|
]
|
||||||
|
|
||||||
|
function hasInteractiveAria(el) {
|
||||||
|
for (let i = 0; i < INTERACTIVE_ARIA_ATTRS.length; i++) {
|
||||||
|
if (el.hasAttribute(INTERACTIVE_ARIA_ATTRS[i])) return true
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
function isInteractiveCandidate(element) {
|
function isInteractiveCandidate(element) {
|
||||||
if (!element || element.nodeType !== Node.ELEMENT_NODE) return false
|
if (!element || element.nodeType !== Node.ELEMENT_NODE) return false
|
||||||
|
|
||||||
@@ -1167,7 +1200,7 @@ export default (
|
|||||||
element.hasAttribute('onclick') ||
|
element.hasAttribute('onclick') ||
|
||||||
element.hasAttribute('role') ||
|
element.hasAttribute('role') ||
|
||||||
element.hasAttribute('tabindex') ||
|
element.hasAttribute('tabindex') ||
|
||||||
element.hasAttribute('aria-') ||
|
hasInteractiveAria(element) ||
|
||||||
element.hasAttribute('data-action') ||
|
element.hasAttribute('data-action') ||
|
||||||
element.getAttribute('contenteditable') === 'true'
|
element.getAttribute('contenteditable') === 'true'
|
||||||
|
|
||||||
@@ -1185,8 +1218,9 @@ export default (
|
|||||||
'details',
|
'details',
|
||||||
'label',
|
'label',
|
||||||
'option',
|
'option',
|
||||||
|
'li',
|
||||||
])
|
])
|
||||||
const INTERACTIVE_ROLES = new Set([
|
const DISTINCT_INTERACTIVE_ROLES = new Set([
|
||||||
'button',
|
'button',
|
||||||
'link',
|
'link',
|
||||||
'menuitem',
|
'menuitem',
|
||||||
@@ -1202,6 +1236,9 @@ export default (
|
|||||||
'searchbox',
|
'searchbox',
|
||||||
'textbox',
|
'textbox',
|
||||||
'listbox',
|
'listbox',
|
||||||
|
'listitem',
|
||||||
|
'treeitem',
|
||||||
|
'row',
|
||||||
'option',
|
'option',
|
||||||
'scrollbar',
|
'scrollbar',
|
||||||
])
|
])
|
||||||
@@ -1278,7 +1315,7 @@ export default (
|
|||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
// Check interactive roles
|
// Check interactive roles
|
||||||
if (role && INTERACTIVE_ROLES.has(role)) {
|
if (role && DISTINCT_INTERACTIVE_ROLES.has(role)) {
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
// Check contenteditable
|
// Check contenteditable
|
||||||
@@ -1297,6 +1334,10 @@ export default (
|
|||||||
if (element.hasAttribute('onclick') || typeof element.onclick === 'function') {
|
if (element.hasAttribute('onclick') || typeof element.onclick === 'function') {
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
// ARIA state attributes imply the element manages its own interaction state
|
||||||
|
if (hasInteractiveAria(element)) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
// return false
|
// return false
|
||||||
|
|
||||||
@@ -1352,6 +1393,12 @@ export default (
|
|||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Scrollable containers are always distinct — the LLM needs their index for targeted scrolling.
|
||||||
|
// Check extraData (already set by isScrollableElement in isInteractiveElement) to avoid redundant layout reads.
|
||||||
|
if (extraData.get(element)?.scrollable) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
// Default to false: if it's interactive but doesn't match above,
|
// Default to false: if it's interactive but doesn't match above,
|
||||||
// assume it triggers the same action as the parent.
|
// assume it triggers the same action as the parent.
|
||||||
return false
|
return false
|
||||||
|
|||||||
@@ -28,8 +28,27 @@ export interface DomConfig {
|
|||||||
includeAttributes?: string[]
|
includeAttributes?: string[]
|
||||||
highlightOpacity?: number
|
highlightOpacity?: number
|
||||||
highlightLabelOpacity?: number
|
highlightLabelOpacity?: number
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Preserve semantic landmark tags in dehydrated output even if not interactive
|
||||||
|
* @note maybe confusing for LLM combining with page scrolling, use with caution
|
||||||
|
**/
|
||||||
|
keepSemanticTags?: boolean
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TODO: corresponding roles
|
||||||
|
const SEMANTIC_TAGS = new Set([
|
||||||
|
'nav',
|
||||||
|
'menu',
|
||||||
|
// 'main',
|
||||||
|
'header',
|
||||||
|
'footer',
|
||||||
|
'aside',
|
||||||
|
// 'article',
|
||||||
|
// 'form',
|
||||||
|
'dialog',
|
||||||
|
])
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 用于检测可交互元素是否是新出现的。
|
* 用于检测可交互元素是否是新出现的。
|
||||||
*/
|
*/
|
||||||
@@ -171,7 +190,11 @@ interface TreeNode {
|
|||||||
*
|
*
|
||||||
* @todo 数据脱敏过滤器
|
* @todo 数据脱敏过滤器
|
||||||
*/
|
*/
|
||||||
export function flatTreeToString(flatTree: FlatDomTree, includeAttributes?: string[]): string {
|
export function flatTreeToString(
|
||||||
|
flatTree: FlatDomTree,
|
||||||
|
includeAttributes: string[] = [],
|
||||||
|
keepSemanticTags = false
|
||||||
|
): string {
|
||||||
const DEFAULT_INCLUDE_ATTRIBUTES = [
|
const DEFAULT_INCLUDE_ATTRIBUTES = [
|
||||||
'title',
|
'title',
|
||||||
'type',
|
'type',
|
||||||
@@ -203,7 +226,7 @@ export function flatTreeToString(flatTree: FlatDomTree, includeAttributes?: stri
|
|||||||
'contenteditable',
|
'contenteditable',
|
||||||
]
|
]
|
||||||
|
|
||||||
const includeAttrs = [...(includeAttributes || []), ...DEFAULT_INCLUDE_ATTRIBUTES]
|
const includeAttrs = [...includeAttributes, ...DEFAULT_INCLUDE_ATTRIBUTES]
|
||||||
|
|
||||||
// Helper function to cap text length
|
// Helper function to cap text length
|
||||||
const capTextLength = (text: string, maxLength: number): string => {
|
const capTextLength = (text: string, maxLength: number): string => {
|
||||||
@@ -294,6 +317,8 @@ export function flatTreeToString(flatTree: FlatDomTree, includeAttributes?: stri
|
|||||||
const depthStr = '\t'.repeat(depth)
|
const depthStr = '\t'.repeat(depth)
|
||||||
|
|
||||||
if (node.type === 'element') {
|
if (node.type === 'element') {
|
||||||
|
const isSemantic = keepSemanticTags && node.tagName && SEMANTIC_TAGS.has(node.tagName)
|
||||||
|
|
||||||
// Add element with highlight_index
|
// Add element with highlight_index
|
||||||
if (node.highlightIndex !== undefined) {
|
if (node.highlightIndex !== undefined) {
|
||||||
nextDepth += 1
|
nextDepth += 1
|
||||||
@@ -391,10 +416,30 @@ export function flatTreeToString(flatTree: FlatDomTree, includeAttributes?: stri
|
|||||||
result.push(line)
|
result.push(line)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Process children regardless
|
// special treatment for semantic tags
|
||||||
|
// even if they are not interactive, we can keep them for clear context
|
||||||
|
|
||||||
|
const emitSemantic = isSemantic && node.highlightIndex === undefined
|
||||||
|
// to check if this tag is empty
|
||||||
|
const mark = emitSemantic ? result.length : -1
|
||||||
|
|
||||||
|
if (emitSemantic) {
|
||||||
|
result.push(`${depthStr}<${node.tagName}>`)
|
||||||
|
nextDepth += 1
|
||||||
|
}
|
||||||
|
|
||||||
for (const child of node.children) {
|
for (const child of node.children) {
|
||||||
processNode(child, nextDepth, result)
|
processNode(child, nextDepth, result)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (emitSemantic) {
|
||||||
|
// empty tag should be removed
|
||||||
|
if (result.length === mark + 1) {
|
||||||
|
result.pop()
|
||||||
|
} else {
|
||||||
|
result.push(`${depthStr}</${node.tagName}>`)
|
||||||
|
}
|
||||||
|
}
|
||||||
} else if (node.type === 'text') {
|
} else if (node.type === 'text') {
|
||||||
// Add text only if it doesn't have a highlighted parent
|
// Add text only if it doesn't have a highlighted parent
|
||||||
if (hasParentWithHighlightIndex(node)) {
|
if (hasParentWithHighlightIndex(node)) {
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ import { isPageDark } from './checkDarkMode'
|
|||||||
import styles from './SimulatorMask.module.css'
|
import styles from './SimulatorMask.module.css'
|
||||||
import cursorStyles from './cursor.module.css'
|
import cursorStyles from './cursor.module.css'
|
||||||
|
|
||||||
export class SimulatorMask {
|
export class SimulatorMask extends EventTarget {
|
||||||
shown: boolean = false
|
shown: boolean = false
|
||||||
wrapper = document.createElement('div')
|
wrapper = document.createElement('div')
|
||||||
motion: Motion | null = null
|
motion: Motion | null = null
|
||||||
@@ -19,6 +19,8 @@ export class SimulatorMask {
|
|||||||
#targetCursorY = 0
|
#targetCursorY = 0
|
||||||
|
|
||||||
constructor() {
|
constructor() {
|
||||||
|
super()
|
||||||
|
|
||||||
this.wrapper.id = 'page-agent-runtime_simulator-mask'
|
this.wrapper.id = 'page-agent-runtime_simulator-mask'
|
||||||
this.wrapper.className = styles.wrapper
|
this.wrapper.className = styles.wrapper
|
||||||
this.wrapper.setAttribute('data-browser-use-ignore', 'true')
|
this.wrapper.setAttribute('data-browser-use-ignore', 'true')
|
||||||
@@ -74,13 +76,34 @@ export class SimulatorMask {
|
|||||||
|
|
||||||
this.#moveCursorToTarget()
|
this.#moveCursorToTarget()
|
||||||
|
|
||||||
window.addEventListener('PageAgent::MovePointerTo', (event: Event) => {
|
// global events
|
||||||
|
// @note Mask should be isolated from the rest of the code.
|
||||||
|
// Global events are easier to manage and cleanup.
|
||||||
|
|
||||||
|
const movePointerToListener = (event: Event) => {
|
||||||
const { x, y } = (event as CustomEvent).detail
|
const { x, y } = (event as CustomEvent).detail
|
||||||
this.setCursorPosition(x, y)
|
this.setCursorPosition(x, y)
|
||||||
})
|
}
|
||||||
|
const clickPointerListener = () => {
|
||||||
window.addEventListener('PageAgent::ClickPointer', (event: Event) => {
|
|
||||||
this.triggerClickAnimation()
|
this.triggerClickAnimation()
|
||||||
|
}
|
||||||
|
const enablePassThroughListener = () => {
|
||||||
|
this.wrapper.style.pointerEvents = 'none'
|
||||||
|
}
|
||||||
|
const disablePassThroughListener = () => {
|
||||||
|
this.wrapper.style.pointerEvents = 'auto'
|
||||||
|
}
|
||||||
|
|
||||||
|
window.addEventListener('PageAgent::MovePointerTo', movePointerToListener)
|
||||||
|
window.addEventListener('PageAgent::ClickPointer', clickPointerListener)
|
||||||
|
window.addEventListener('PageAgent::EnablePassThrough', enablePassThroughListener)
|
||||||
|
window.addEventListener('PageAgent::DisablePassThrough', disablePassThroughListener)
|
||||||
|
|
||||||
|
this.addEventListener('dispose', () => {
|
||||||
|
window.removeEventListener('PageAgent::MovePointerTo', movePointerToListener)
|
||||||
|
window.removeEventListener('PageAgent::ClickPointer', clickPointerListener)
|
||||||
|
window.removeEventListener('PageAgent::EnablePassThrough', enablePassThroughListener)
|
||||||
|
window.removeEventListener('PageAgent::DisablePassThrough', disablePassThroughListener)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -177,7 +200,9 @@ export class SimulatorMask {
|
|||||||
}
|
}
|
||||||
|
|
||||||
dispose() {
|
dispose() {
|
||||||
|
console.log('dispose SimulatorMask')
|
||||||
this.motion?.dispose()
|
this.motion?.dispose()
|
||||||
this.wrapper.remove()
|
this.wrapper.remove()
|
||||||
|
this.dispatchEvent(new Event('dispose'))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -48,15 +48,33 @@ export async function waitFor(seconds: number): Promise<void> {
|
|||||||
await new Promise((resolve) => setTimeout(resolve, seconds * 1000))
|
await new Promise((resolve) => setTimeout(resolve, seconds * 1000))
|
||||||
}
|
}
|
||||||
|
|
||||||
// ======= dom utils =======
|
// ======= mask events =======
|
||||||
|
|
||||||
export async function movePointerToElement(element: HTMLElement) {
|
/**
|
||||||
const rect = element.getBoundingClientRect()
|
* Move the visual pointer to a position within an element.
|
||||||
|
* @param x - x coordinate in the element's document viewport
|
||||||
|
* @param y - y coordinate in the element's document viewport
|
||||||
|
*/
|
||||||
|
export async function movePointerToElement(element: HTMLElement, x: number, y: number) {
|
||||||
const offset = getIframeOffset(element)
|
const offset = getIframeOffset(element)
|
||||||
const x = rect.left + rect.width / 2 + offset.x
|
|
||||||
const y = rect.top + rect.height / 2 + offset.y
|
|
||||||
|
|
||||||
window.dispatchEvent(new CustomEvent('PageAgent::MovePointerTo', { detail: { x, y } }))
|
window.dispatchEvent(
|
||||||
|
new CustomEvent('PageAgent::MovePointerTo', {
|
||||||
|
detail: { x: x + offset.x, y: y + offset.y },
|
||||||
|
})
|
||||||
|
)
|
||||||
|
|
||||||
await waitFor(0.3)
|
await waitFor(0.3)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export async function clickPointer() {
|
||||||
|
window.dispatchEvent(new CustomEvent('PageAgent::ClickPointer'))
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function enablePassThrough() {
|
||||||
|
window.dispatchEvent(new CustomEvent('PageAgent::EnablePassThrough'))
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function disablePassThrough() {
|
||||||
|
window.dispatchEvent(new CustomEvent('PageAgent::DisablePassThrough'))
|
||||||
|
}
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@page-agent/ui",
|
"name": "@page-agent/ui",
|
||||||
"version": "1.6.1",
|
"version": "1.7.1",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "./dist/lib/page-agent-ui.js",
|
"main": "./dist/lib/page-agent-ui.js",
|
||||||
"module": "./dist/lib/page-agent-ui.js",
|
"module": "./dist/lib/page-agent-ui.js",
|
||||||
|
|||||||
@@ -369,6 +369,7 @@ export class Panel {
|
|||||||
}
|
}
|
||||||
|
|
||||||
#createWrapper(): HTMLElement {
|
#createWrapper(): HTMLElement {
|
||||||
|
const taskInputMaxLength = 1000
|
||||||
const wrapper = document.createElement('div')
|
const wrapper = document.createElement('div')
|
||||||
wrapper.id = 'page-agent-runtime_agent-panel'
|
wrapper.id = 'page-agent-runtime_agent-panel'
|
||||||
wrapper.className = styles.wrapper
|
wrapper.className = styles.wrapper
|
||||||
@@ -406,7 +407,7 @@ export class Panel {
|
|||||||
<input
|
<input
|
||||||
type="text"
|
type="text"
|
||||||
class="${styles.taskInput}"
|
class="${styles.taskInput}"
|
||||||
maxlength="200"
|
maxlength="${taskInputMaxLength}"
|
||||||
/>
|
/>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -85,7 +85,7 @@ Uses wouter browser routing with base path for GitHub Pages deployment at `https
|
|||||||
```tsx
|
```tsx
|
||||||
// main.tsx
|
// main.tsx
|
||||||
<Router base="/page-agent">
|
<Router base="/page-agent">
|
||||||
<PagesRouter />
|
<PagesRouter />
|
||||||
</Router>
|
</Router>
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -109,6 +109,7 @@ Instead of `404.html` redirects, the build copies `index.html` into every route
|
|||||||
2. Add route in `src/pages/docs/index.tsx`
|
2. Add route in `src/pages/docs/index.tsx`
|
||||||
3. Add navigation item in `src/pages/docs/Layout.tsx`
|
3. Add navigation item in `src/pages/docs/Layout.tsx`
|
||||||
4. Add path to `SPA_ROUTES` in `vite.config.js`
|
4. Add path to `SPA_ROUTES` in `vite.config.js`
|
||||||
|
5. Keep `<slug>` consistent across the folder name, import path, route path, sidebar link, and any related cross-links. If the public route changes, rename the folder too.
|
||||||
|
|
||||||
## Configuration Files
|
## Configuration Files
|
||||||
|
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
{
|
{
|
||||||
"name": "@page-agent/website",
|
"name": "@page-agent/website",
|
||||||
"private": true,
|
"private": true,
|
||||||
"version": "1.6.1",
|
"version": "1.7.1",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"dev": "vite --host 0.0.0.0",
|
"dev": "vite --host 0.0.0.0",
|
||||||
@@ -19,13 +19,13 @@
|
|||||||
"@types/react-dom": "^19.2.1",
|
"@types/react-dom": "^19.2.1",
|
||||||
"class-variance-authority": "^0.7.1",
|
"class-variance-authority": "^0.7.1",
|
||||||
"clsx": "^2.1.1",
|
"clsx": "^2.1.1",
|
||||||
"lucide-react": "^0.577.0",
|
"lucide-react": "^1.7.0",
|
||||||
"motion": "^12.37.0",
|
"motion": "^12.38.0",
|
||||||
"next-themes": "^0.4.6",
|
"next-themes": "^0.4.6",
|
||||||
"react": "^19.2.4",
|
"react": "^19.2.4",
|
||||||
"react-dom": "^19.2.4",
|
"react-dom": "^19.2.4",
|
||||||
"rough-notation": "^0.5.1",
|
"rough-notation": "^0.5.1",
|
||||||
"simple-icons": "^16.12.0",
|
"simple-icons": "^16.14.0",
|
||||||
"sonner": "^2.0.7",
|
"sonner": "^2.0.7",
|
||||||
"tailwind-merge": "^3.5.0",
|
"tailwind-merge": "^3.5.0",
|
||||||
"tailwindcss": "^4.1.14",
|
"tailwindcss": "^4.1.14",
|
||||||
|
|||||||
@@ -18,7 +18,7 @@ export default function Header() {
|
|||||||
return (
|
return (
|
||||||
<>
|
<>
|
||||||
<header
|
<header
|
||||||
className="relative z-50 bg-white/80 dark:bg-gray-900/80 backdrop-blur-md border-b border-gray-200 dark:border-gray-700"
|
className="sticky top-0 z-50 bg-white/80 dark:bg-gray-900/80 backdrop-blur-md border-b border-gray-200 dark:border-gray-700"
|
||||||
role="banner"
|
role="banner"
|
||||||
>
|
>
|
||||||
<div className="max-w-7xl mx-auto px-6 py-4">
|
<div className="max-w-7xl mx-auto px-6 py-4">
|
||||||
|
|||||||
@@ -1,8 +1,8 @@
|
|||||||
// Demo build (auto-init with demo LLM, for quick testing)
|
// Demo build (auto-init with demo LLM, for quick testing)
|
||||||
export const CDN_DEMO_URL =
|
export const CDN_DEMO_URL =
|
||||||
'https://cdn.jsdelivr.net/npm/page-agent@1.6.1/dist/iife/page-agent.demo.js'
|
'https://cdn.jsdelivr.net/npm/page-agent@1.7.1/dist/iife/page-agent.demo.js'
|
||||||
export const CDN_DEMO_CN_URL =
|
export const CDN_DEMO_CN_URL =
|
||||||
'https://registry.npmmirror.com/page-agent/1.6.1/files/dist/iife/page-agent.demo.js'
|
'https://registry.npmmirror.com/page-agent/1.7.1/files/dist/iife/page-agent.demo.js'
|
||||||
|
|
||||||
// Demo LLM for website testing (homepage quick trial uses flash)
|
// Demo LLM for website testing (homepage quick trial uses flash)
|
||||||
export const DEMO_MODEL = 'qwen3.5-flash'
|
export const DEMO_MODEL = 'qwen3.5-flash'
|
||||||
|
|||||||
@@ -100,7 +100,7 @@ console.log(result.history) // Full execution history`}
|
|||||||
>
|
>
|
||||||
AgentConfig
|
AgentConfig
|
||||||
</Link>{' '}
|
</Link>{' '}
|
||||||
和{' '}
|
、 PanelConfig 和{' '}
|
||||||
<Link
|
<Link
|
||||||
href="/advanced/page-controller#configuration"
|
href="/advanced/page-controller#configuration"
|
||||||
className="text-blue-600 dark:text-blue-400 hover:underline"
|
className="text-blue-600 dark:text-blue-400 hover:underline"
|
||||||
@@ -125,7 +125,7 @@ console.log(result.history) // Full execution history`}
|
|||||||
>
|
>
|
||||||
AgentConfig
|
AgentConfig
|
||||||
</Link>{' '}
|
</Link>{' '}
|
||||||
and{' '}
|
, PanelConfig, and{' '}
|
||||||
<Link
|
<Link
|
||||||
href="/advanced/page-controller#configuration"
|
href="/advanced/page-controller#configuration"
|
||||||
className="text-blue-600 dark:text-blue-400 hover:underline"
|
className="text-blue-600 dark:text-blue-400 hover:underline"
|
||||||
|
|||||||
@@ -108,6 +108,14 @@ const agent = new PageAgentCore({
|
|||||||
? '在 DOM 提取中包含的额外 HTML 属性。支持通配符 *(如 data-* 匹配所有 data- 开头的属性)。默认已包含常见属性如 role, aria-label 等。'
|
? '在 DOM 提取中包含的额外 HTML 属性。支持通配符 *(如 data-* 匹配所有 data- 开头的属性)。默认已包含常见属性如 role, aria-label 等。'
|
||||||
: 'Additional HTML attributes to include in DOM extraction. Supports wildcard * (e.g. data-* matches all data- prefixed attributes). Common attributes like role, aria-label are included by default.',
|
: 'Additional HTML attributes to include in DOM extraction. Supports wildcard * (e.g. data-* matches all data- prefixed attributes). Common attributes like role, aria-label are included by default.',
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
name: 'keepSemanticTags',
|
||||||
|
type: 'boolean',
|
||||||
|
defaultValue: 'false',
|
||||||
|
description: isZh
|
||||||
|
? '在简化输出中保留语义标签(如 nav, main, header, footer, aside 等),即使它们不可交互。帮助 LLM 理解页面结构。'
|
||||||
|
: 'Preserve semantic landmark tags (e.g. nav, main, header, footer, aside) in dehydrated output even if not interactive. Helps LLM understand page structure.',
|
||||||
|
},
|
||||||
]}
|
]}
|
||||||
/>
|
/>
|
||||||
</section>
|
</section>
|
||||||
|
|||||||
@@ -199,7 +199,9 @@ interface ExecuteConfig {
|
|||||||
model: string // Model name
|
model: string // Model name
|
||||||
apiKey?: string // LLM AK
|
apiKey?: string // LLM AK
|
||||||
|
|
||||||
|
systemInstruction?: string // Global system-level instructions
|
||||||
includeInitialTab?: boolean
|
includeInitialTab?: boolean
|
||||||
|
experimentalIncludeAllTabs?: boolean // Control all unpinned tabs in the window
|
||||||
onStatusChange?: (status: AgentStatus) => void
|
onStatusChange?: (status: AgentStatus) => void
|
||||||
onActivity?: (activity: AgentActivity) => void
|
onActivity?: (activity: AgentActivity) => void
|
||||||
onHistoryUpdate?: (history: HistoricalEvent[]) => void
|
onHistoryUpdate?: (history: HistoricalEvent[]) => void
|
||||||
@@ -233,6 +235,7 @@ const result = await window.PAGE_AGENT_EXT.execute(
|
|||||||
apiKey: 'your-api-key',
|
apiKey: 'your-api-key',
|
||||||
model: 'gpt-5.2',
|
model: 'gpt-5.2',
|
||||||
// includeInitialTab: false, // 设为 false 排除初始标签页
|
// includeInitialTab: false, // 设为 false 排除初始标签页
|
||||||
|
// experimentalIncludeAllTabs: true, // 控制窗口内所有非固定标签页
|
||||||
onStatusChange: status => console.log('状态变化:', status),
|
onStatusChange: status => console.log('状态变化:', status),
|
||||||
onActivity: activity => console.log('活动:', activity),
|
onActivity: activity => console.log('活动:', activity),
|
||||||
onHistoryUpdate: history => console.log('历史更新:', history)
|
onHistoryUpdate: history => console.log('历史更新:', history)
|
||||||
@@ -248,6 +251,7 @@ const result = await window.PAGE_AGENT_EXT.execute(
|
|||||||
apiKey: 'your-api-key',
|
apiKey: 'your-api-key',
|
||||||
model: 'gpt-5.2',
|
model: 'gpt-5.2',
|
||||||
// includeInitialTab: false, // Set to false to exclude initial tab
|
// includeInitialTab: false, // Set to false to exclude initial tab
|
||||||
|
// experimentalIncludeAllTabs: true, // Control all unpinned tabs in the window
|
||||||
onStatusChange: status => console.log('Status change:', status),
|
onStatusChange: status => console.log('Status change:', status),
|
||||||
onActivity: activity => console.log('Activity:', activity),
|
onActivity: activity => console.log('Activity:', activity),
|
||||||
onHistoryUpdate: history => console.log('History update:', history)
|
onHistoryUpdate: history => console.log('History update:', history)
|
||||||
|
|||||||
@@ -16,12 +16,12 @@ const BASELINE = new Set([
|
|||||||
// Models grouped by brand, newest first
|
// Models grouped by brand, newest first
|
||||||
const MODEL_GROUPS: Record<string, string[]> = {
|
const MODEL_GROUPS: Record<string, string[]> = {
|
||||||
Qwen: [
|
Qwen: [
|
||||||
|
'qwen3.6-plus',
|
||||||
'qwen3.5-plus',
|
'qwen3.5-plus',
|
||||||
'qwen3.5-flash',
|
'qwen3.5-flash',
|
||||||
'qwen3-coder-next',
|
'qwen3-coder-next',
|
||||||
'qwen-3-max',
|
'qwen-3-max',
|
||||||
'qwen-3-plus',
|
'qwen-3-plus',
|
||||||
'qwen3:14b (ollama)',
|
|
||||||
],
|
],
|
||||||
OpenAI: ['gpt-5.4', 'gpt-5.2', 'gpt-5.1', 'gpt-5', 'gpt-5-mini', 'gpt-4.1', 'gpt-4.1-mini'],
|
OpenAI: ['gpt-5.4', 'gpt-5.2', 'gpt-5.1', 'gpt-5', 'gpt-5-mini', 'gpt-4.1', 'gpt-4.1-mini'],
|
||||||
DeepSeek: ['deepseek-3.2'],
|
DeepSeek: ['deepseek-3.2'],
|
||||||
@@ -64,7 +64,6 @@ export default function Models() {
|
|||||||
: 'Supports models that comply with OpenAI API specification and support tool calls, including public cloud services and private deployments.'}
|
: 'Supports models that comply with OpenAI API specification and support tool calls, including public cloud services and private deployments.'}
|
||||||
</p>
|
</p>
|
||||||
|
|
||||||
{/* Models Section */}
|
|
||||||
<section className="mb-10">
|
<section className="mb-10">
|
||||||
<Heading id="tested-models" className="text-2xl font-semibold mb-3">
|
<Heading id="tested-models" className="text-2xl font-semibold mb-3">
|
||||||
{isZh ? '已测试模型' : 'Tested Models'}
|
{isZh ? '已测试模型' : 'Tested Models'}
|
||||||
@@ -87,9 +86,10 @@ export default function Models() {
|
|||||||
</div>
|
</div>
|
||||||
</section>
|
</section>
|
||||||
|
|
||||||
{/* Tips Section */}
|
|
||||||
<section className="mb-10">
|
<section className="mb-10">
|
||||||
<h2 className="text-2xl font-semibold mb-4">{isZh ? '提示' : 'Tips'}</h2>
|
<Heading id="tips" level={3}>
|
||||||
|
Tips
|
||||||
|
</Heading>
|
||||||
<div className="p-4 bg-blue-50 dark:bg-blue-950/20 rounded-lg border border-blue-200 dark:border-blue-800">
|
<div className="p-4 bg-blue-50 dark:bg-blue-950/20 rounded-lg border border-blue-200 dark:border-blue-800">
|
||||||
<ul className="text-sm text-gray-700 dark:text-gray-300 space-y-2 list-disc pl-5">
|
<ul className="text-sm text-gray-700 dark:text-gray-300 space-y-2 list-disc pl-5">
|
||||||
<li>
|
<li>
|
||||||
@@ -111,7 +111,6 @@ export default function Models() {
|
|||||||
</div>
|
</div>
|
||||||
</section>
|
</section>
|
||||||
|
|
||||||
{/* Configuration Section */}
|
|
||||||
<section className="mb-10">
|
<section className="mb-10">
|
||||||
<Heading id="configuration">{isZh ? '配置方式' : 'Configuration'}</Heading>
|
<Heading id="configuration">{isZh ? '配置方式' : 'Configuration'}</Heading>
|
||||||
<CodeEditor
|
<CodeEditor
|
||||||
@@ -121,25 +120,50 @@ const pageAgent = new PageAgent({
|
|||||||
apiKey: 'your-api-key',
|
apiKey: 'your-api-key',
|
||||||
model: 'qwen3.5-plus'
|
model: 'qwen3.5-plus'
|
||||||
});
|
});
|
||||||
|
|
||||||
// MiniMax
|
|
||||||
const pageAgent = new PageAgent({
|
|
||||||
baseURL: 'https://api.minimax.io/v1',
|
|
||||||
apiKey: 'your-minimax-api-key',
|
|
||||||
model: 'MiniMax-M2.7'
|
|
||||||
});
|
|
||||||
|
|
||||||
// Self-hosted models (e.g., Ollama) — no apiKey needed
|
|
||||||
const pageAgent = new PageAgent({
|
|
||||||
baseURL: 'http://localhost:11434/v1',
|
|
||||||
model: 'qwen3:14b'
|
|
||||||
});
|
|
||||||
|
|
||||||
`}
|
`}
|
||||||
/>
|
/>
|
||||||
</section>
|
</section>
|
||||||
|
|
||||||
{/* Free Testing API Section */}
|
<section className="mb-10">
|
||||||
|
<Heading id="production-authentication" className="text-2xl font-semibold mb-4">
|
||||||
|
{isZh ? '🔐 生产环境鉴权' : '🔐 Production Authentication'}
|
||||||
|
</Heading>
|
||||||
|
<p className="text-sm text-gray-600 dark:text-gray-400 mb-3">
|
||||||
|
{isZh
|
||||||
|
? '如果你只是将它用作个人助手,可以直接连接你的 LLM 服务。'
|
||||||
|
: 'If you only use it as a personal assistant, you can connect to your LLM service directly.'}
|
||||||
|
</p>
|
||||||
|
<p className="text-sm text-gray-600 dark:text-gray-400 mb-3">
|
||||||
|
{isZh ? (
|
||||||
|
<>
|
||||||
|
如果你计划将它集成到你的 Web 应用中,建议搭建一个后端代理来转发 LLM 请求,并使用{' '}
|
||||||
|
<code>customFetch</code> 携带 Cookie 或其他鉴权信息:
|
||||||
|
</>
|
||||||
|
) : (
|
||||||
|
<>
|
||||||
|
If you plan to integrate it into your web app, it's better to have a backend proxy for
|
||||||
|
the LLM and use <code>customFetch</code> to authenticate the request with cookies or
|
||||||
|
other methods:
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
</p>
|
||||||
|
<CodeEditor
|
||||||
|
code={`const agent = new PageAgent({
|
||||||
|
baseURL: '/api/llm-proxy',
|
||||||
|
model: 'gpt-5.1',
|
||||||
|
customFetch: (url, init) =>
|
||||||
|
fetch(url, { ...init, credentials: 'include' }),
|
||||||
|
});`}
|
||||||
|
/>
|
||||||
|
<div className="mt-4 bg-yellow-50 dark:bg-yellow-950/20 border-l-4 border-yellow-500 p-4 rounded-r-lg">
|
||||||
|
<p className="text-sm font-semibold text-yellow-900 dark:text-yellow-200">
|
||||||
|
{isZh
|
||||||
|
? '⚠️ 永远不要把真实的 LLM API Key 提交到前端代码中'
|
||||||
|
: '⚠️ NEVER commit real LLM API keys to your frontend code'}
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
|
||||||
<section className="mb-10">
|
<section className="mb-10">
|
||||||
<Heading id="free-testing-api">{isZh ? '免费测试接口' : 'Free Testing API'}</Heading>
|
<Heading id="free-testing-api">{isZh ? '免费测试接口' : 'Free Testing API'}</Heading>
|
||||||
<p className="text-sm text-gray-600 dark:text-gray-400 mb-4">
|
<p className="text-sm text-gray-600 dark:text-gray-400 mb-4">
|
||||||
@@ -181,115 +205,143 @@ const pageAgent = new PageAgent({
|
|||||||
</a>
|
</a>
|
||||||
</p>
|
</p>
|
||||||
<CodeEditor
|
<CodeEditor
|
||||||
code={`# qwen3.5-plus (default for demos) or qwen3.5-flash (lighter)
|
code={`# qwen3.5-plus / qwen3.5-flash
|
||||||
LLM_BASE_URL="https://page-ag-testing-ohftxirgbn.cn-shanghai.fcapp.run"
|
LLM_BASE_URL="https://page-ag-testing-ohftxirgbn.cn-shanghai.fcapp.run"
|
||||||
LLM_MODEL_NAME="qwen3.5-plus"
|
LLM_MODEL_NAME="qwen3.5-plus"`}
|
||||||
LLM_API_KEY="NA"`}
|
|
||||||
/>
|
/>
|
||||||
</div>
|
</div>
|
||||||
</section>
|
</section>
|
||||||
|
|
||||||
{/* Ollama Section */}
|
|
||||||
<section className="mb-10">
|
<section className="mb-10">
|
||||||
<Heading id="ollama">Ollama</Heading>
|
<Heading id="local-runtimes">{isZh ? '本地运行时' : 'Local Runtimes'}</Heading>
|
||||||
<p className="text-sm text-gray-600 dark:text-gray-400 mb-4">
|
<p className="text-sm text-gray-600 dark:text-gray-400 mb-6">
|
||||||
{isZh
|
{isZh
|
||||||
? '已在 Ollama 0.15 + qwen3:14b (RTX3090 24GB) 上测试通过。'
|
? '通过 Ollama、LM Studio 等本地 OpenAI-compatible 运行时接入 PageAgent,实现离线或局域网部署。'
|
||||||
: 'Tested on Ollama 0.15 with qwen3:14b (RTX3090 24GB).'}
|
: 'Use local OpenAI-compatible runtimes such as Ollama and LM Studio with PageAgent for offline or LAN deployments.'}
|
||||||
</p>
|
</p>
|
||||||
<CodeEditor
|
|
||||||
code={`LLM_BASE_URL="http://localhost:11434/v1"
|
|
||||||
LLM_API_KEY="NA"
|
|
||||||
LLM_MODEL_NAME="qwen3:14b"`}
|
|
||||||
/>
|
|
||||||
<div className="mt-4 p-4 bg-amber-50 dark:bg-amber-950/20 rounded-lg border border-amber-200 dark:border-amber-800">
|
|
||||||
<h3 className="font-semibold text-amber-900 dark:text-amber-200 mb-2">
|
|
||||||
{isZh ? '⚠️ 注意事项' : '⚠️ Important Notes'}
|
|
||||||
</h3>
|
|
||||||
<ul className="text-sm text-gray-700 dark:text-gray-300 space-y-2 list-disc pl-5">
|
|
||||||
<li>
|
|
||||||
{isZh
|
|
||||||
? '确保 OLLAMA_ORIGINS 设置为 * 以避免 403 错误'
|
|
||||||
: 'Add * to OLLAMA_ORIGINS to avoid 403 errors'}
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
{isZh
|
|
||||||
? '小于 10B 参数的模型通常效果不佳'
|
|
||||||
: 'Models smaller than 10B are unlikely to be strong enough'}
|
|
||||||
</li>
|
|
||||||
<li>{isZh ? '需要支持 tool_call 的模型' : 'Requires tool_call capable models'}</li>
|
|
||||||
<li>
|
|
||||||
{isZh
|
|
||||||
? '确保上下文长度大于输入 token 数,否则 Ollama 会静默截断 prompt。普通页面约需 15k token,随步骤增加。默认 4k 上下文长度无法正常工作'
|
|
||||||
: 'Ensure context length exceeds input tokens, or Ollama will silently truncate prompts. ~15k tokens for a typical page, increases with steps. Default 4k context length will NOT work'}
|
|
||||||
</li>
|
|
||||||
</ul>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div className="mt-4">
|
<div className="space-y-10">
|
||||||
<h3 className="font-semibold text-gray-900 dark:text-gray-100 mb-3">
|
<section>
|
||||||
{isZh ? '建议启动参数' : 'Recommended Startup'}
|
<Heading id="requirements" level={3}>
|
||||||
</h3>
|
Requirements
|
||||||
<p className="text-sm text-gray-600 dark:text-gray-400 mb-3">
|
</Heading>
|
||||||
{isZh
|
<div className="mt-4 p-5 bg-amber-50 dark:bg-amber-950/20 rounded-xl border border-amber-200 dark:border-amber-800">
|
||||||
? '启动 Ollama 时建议配置以下环境变量:扩大上下文窗口、允许跨域访问、监听所有网络接口。'
|
<ul className="text-sm text-gray-700 dark:text-gray-300 space-y-2 list-disc pl-5">
|
||||||
: 'Start Ollama with these environment variables: larger context window, allow cross-origin access, and listen on all interfaces.'}
|
<li>
|
||||||
</p>
|
{isZh
|
||||||
|
? '务必打开 CORS,否则浏览器无法直接请求本地 LLM 服务。'
|
||||||
|
: 'Enable CORS, otherwise the browser cannot call your local LLM endpoint directly.'}
|
||||||
|
</li>
|
||||||
|
<li>
|
||||||
|
{isZh
|
||||||
|
? '将 context length 或 content length 至少设置为 8000。普通页面常常需要 15k token 左右,默认 4k 很容易被截断。'
|
||||||
|
: 'Set context length or content length to at least 8000. A typical page often needs around 15k tokens, so the default 4k usually truncates prompts.'}
|
||||||
|
</li>
|
||||||
|
<li>
|
||||||
|
{isZh ? '需要支持 tool_call 的模型。' : 'Use a model with tool_call support.'}
|
||||||
|
</li>
|
||||||
|
<li>
|
||||||
|
{isZh
|
||||||
|
? '小于 10B 参数的模型通常效果不佳。'
|
||||||
|
: 'Models smaller than 10B are usually not strong enough.'}
|
||||||
|
</li>
|
||||||
|
</ul>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
|
||||||
<div className="space-y-2">
|
<section>
|
||||||
<p className="text-xs font-medium text-gray-500 dark:text-gray-400">macOS / Linux</p>
|
<Heading id="local-configuration" level={3}>
|
||||||
|
{isZh ? '基础配置' : 'Basic Configuration'}
|
||||||
|
</Heading>
|
||||||
<CodeEditor
|
<CodeEditor
|
||||||
code={`OLLAMA_CONTEXT_LENGTH=64000 OLLAMA_HOST=0.0.0.0:11434 OLLAMA_ORIGINS="*" ollama serve`}
|
code={`// Local OpenAI-compatible runtime - no apiKey needed
|
||||||
/>
|
const pageAgent = new PageAgent({
|
||||||
|
baseURL: 'http://localhost:11434/v1',
|
||||||
|
model: 'qwen3:14b'
|
||||||
|
});
|
||||||
|
|
||||||
<p className="text-xs font-medium text-gray-500 dark:text-gray-400 pt-2">
|
// Or connect to LM Studio
|
||||||
Windows (PowerShell)
|
const lmStudioAgent = new PageAgent({
|
||||||
|
baseURL: 'http://127.0.0.1:1234/v1',
|
||||||
|
model: 'qwen/qwen3.5-27b'
|
||||||
|
});
|
||||||
|
`}
|
||||||
|
/>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<section>
|
||||||
|
<Heading id="ollama" level={3}>
|
||||||
|
Ollama
|
||||||
|
</Heading>
|
||||||
|
<p className="text-sm text-gray-600 dark:text-gray-400 mb-4">
|
||||||
|
{isZh
|
||||||
|
? '已在 Ollama 0.15 + qwen3:14b (RTX3090 24GB) 上测试通过。'
|
||||||
|
: 'Tested on Ollama 0.15 with qwen3:14b (RTX3090 24GB).'}
|
||||||
</p>
|
</p>
|
||||||
<CodeEditor
|
<CodeEditor
|
||||||
code={`$env:OLLAMA_CONTEXT_LENGTH=64000; $env:OLLAMA_HOST="0.0.0.0:11434"; $env:OLLAMA_ORIGINS="*"; ollama serve`}
|
code={`LLM_BASE_URL="http://localhost:11434/v1"
|
||||||
|
LLM_MODEL_NAME="qwen3:14b"`}
|
||||||
/>
|
/>
|
||||||
</div>
|
<div className="mt-4 p-4 bg-amber-50 dark:bg-amber-950/20 rounded-lg border border-amber-200 dark:border-amber-800">
|
||||||
</div>
|
<h3 className="font-semibold text-amber-900 dark:text-amber-200 mb-2">
|
||||||
</section>
|
{isZh ? '⚠️ 注意事项' : '⚠️ Important Notes'}
|
||||||
|
</h3>
|
||||||
|
<p className="text-sm text-gray-700 dark:text-gray-300">
|
||||||
|
{isZh
|
||||||
|
? '如果浏览器侧请求失败,优先检查 Ollama 是否已按上面的要求开启 CORS。'
|
||||||
|
: 'If browser-side requests fail, check whether Ollama has CORS enabled as required above.'}
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
|
||||||
{/* Production Authentication */}
|
<div className="mt-4">
|
||||||
<section className="mb-10">
|
<h3 className="font-semibold text-gray-900 dark:text-gray-100 mb-3">
|
||||||
<Heading id="production-authentication" className="text-2xl font-semibold mb-4">
|
{isZh ? '建议启动参数' : 'Recommended Startup'}
|
||||||
{isZh ? '🔐 生产环境鉴权' : '🔐 Production Authentication'}
|
</h3>
|
||||||
</Heading>
|
<p className="text-sm text-gray-600 dark:text-gray-400 mb-3">
|
||||||
<p className="text-sm text-gray-600 dark:text-gray-400 mb-3">
|
{isZh
|
||||||
{isZh
|
? '启动 Ollama 时建议同时放大上下文窗口并开启跨域访问。'
|
||||||
? '如果你只是将它用作个人助手,可以直接连接你的 LLM 服务。'
|
: 'When starting Ollama, increase the context window and enable cross-origin access.'}
|
||||||
: 'If you only use it as a personal assistant, you can connect to your LLM service directly.'}
|
</p>
|
||||||
</p>
|
|
||||||
<p className="text-sm text-gray-600 dark:text-gray-400 mb-3">
|
<div className="space-y-2">
|
||||||
{isZh ? (
|
<p className="text-xs font-medium text-gray-500 dark:text-gray-400">
|
||||||
<>
|
macOS / Linux
|
||||||
如果你计划将它集成到你的 Web 应用中,建议搭建一个后端代理来转发 LLM 请求,并使用{' '}
|
</p>
|
||||||
<code>customFetch</code> 携带 Cookie 或其他鉴权信息:
|
<CodeEditor
|
||||||
</>
|
code={`OLLAMA_CONTEXT_LENGTH=64000 OLLAMA_HOST=0.0.0.0:11434 OLLAMA_ORIGINS="*" ollama serve`}
|
||||||
) : (
|
/>
|
||||||
<>
|
|
||||||
If you plan to integrate it into your web app, it's better to have a backend proxy for
|
<p className="text-xs font-medium text-gray-500 dark:text-gray-400 pt-2">
|
||||||
the LLM and use <code>customFetch</code> to authenticate the request with cookies or
|
Windows (PowerShell)
|
||||||
other methods:
|
</p>
|
||||||
</>
|
<CodeEditor
|
||||||
)}
|
code={`$env:OLLAMA_CONTEXT_LENGTH=64000; $env:OLLAMA_HOST="0.0.0.0:11434"; $env:OLLAMA_ORIGINS="*"; ollama serve`}
|
||||||
</p>
|
/>
|
||||||
<CodeEditor
|
</div>
|
||||||
code={`const agent = new PageAgent({
|
</div>
|
||||||
baseURL: '/api/llm-proxy',
|
</section>
|
||||||
model: 'gpt-5.1',
|
|
||||||
customFetch: (url, init) =>
|
<section>
|
||||||
fetch(url, { ...init, credentials: 'include' }),
|
<Heading id="lm-studio" level={3}>
|
||||||
});`}
|
LM Studio
|
||||||
/>
|
</Heading>
|
||||||
<div className="mt-4 bg-yellow-50 dark:bg-yellow-950/20 border-l-4 border-yellow-500 p-4 rounded-r-lg">
|
<CodeEditor
|
||||||
<p className="text-sm font-semibold text-yellow-900 dark:text-yellow-200">
|
code={`LLM_BASE_URL="http://127.0.0.1:1234/v1"
|
||||||
{isZh
|
LLM_MODEL_NAME="qwen/qwen3.5-27b"`}
|
||||||
? '⚠️ 永远不要把真实的 LLM API Key 提交到前端代码中'
|
/>
|
||||||
: '⚠️ NEVER commit real LLM API keys to your frontend code'}
|
<div className="mt-4 p-4 bg-amber-50 dark:bg-amber-950/20 rounded-lg border border-amber-200 dark:border-amber-800">
|
||||||
</p>
|
<h3 className="font-semibold text-amber-900 dark:text-amber-200 mb-2">
|
||||||
|
{isZh ? '⚠️ 注意事项' : '⚠️ Important Notes'}
|
||||||
|
</h3>
|
||||||
|
<ul className="text-sm text-gray-700 dark:text-gray-300 space-y-2 list-disc pl-5">
|
||||||
|
<li>
|
||||||
|
{isZh
|
||||||
|
? 'Agent 必须启用 disableNamedToolChoice,否则 tool_choice 参数会报错。'
|
||||||
|
: 'Enable disableNamedToolChoice in the agent config, otherwise the tool_choice parameter may fail.'}
|
||||||
|
</li>
|
||||||
|
</ul>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
</div>
|
</div>
|
||||||
</section>
|
</section>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -274,10 +274,12 @@ function ApiErrorsContent(isZh: boolean) {
|
|||||||
code={`const agent = new PageAgent({
|
code={`const agent = new PageAgent({
|
||||||
// ...
|
// ...
|
||||||
customFetch: async (url, init) => {
|
customFetch: async (url, init) => {
|
||||||
const body = JSON.parse(init.body)
|
|
||||||
// Adapt parameters for your provider
|
// Adapt parameters for your provider
|
||||||
delete body.stream_options
|
const body = JSON.parse(init.body)
|
||||||
return fetch(url, { ...init, body: JSON.stringify(body) })
|
delete body.tool_choice
|
||||||
|
const bodyStr = JSON.stringify(body)
|
||||||
|
|
||||||
|
return fetch(url, { ...init, body: bodyStr })
|
||||||
},
|
},
|
||||||
})`}
|
})`}
|
||||||
/>
|
/>
|
||||||
|
|||||||
@@ -16,12 +16,13 @@ const LLM_CLOUD: {
|
|||||||
opacity: number
|
opacity: number
|
||||||
}[] = [
|
}[] = [
|
||||||
{ name: 'OpenAI', color: '#10b981', x: 18, y: 22, size: 1.5, opacity: 1 },
|
{ name: 'OpenAI', color: '#10b981', x: 18, y: 22, size: 1.5, opacity: 1 },
|
||||||
{ name: 'Claude', color: '#f97316', x: 62, y: 15, size: 1.35, opacity: 0.95 },
|
{ name: 'Claude', color: '#f97316', x: 58, y: 15, size: 1.35, opacity: 0.95 },
|
||||||
{ name: 'Qwen', color: '#8b5cf6', x: 38, y: 50, size: 1.8, opacity: 0.9 },
|
{ name: 'Qwen', color: '#8b5cf6', x: 35, y: 50, size: 1.8, opacity: 0.9 },
|
||||||
{ name: 'Gemini', color: '#3b82f6', x: 68, y: 48, size: 1.2, opacity: 0.85 },
|
{ name: 'Gemini', color: '#3b82f6', x: 72, y: 48, size: 1.2, opacity: 0.85 },
|
||||||
{ name: 'DeepSeek', color: '#06b6d4', x: 10, y: 65, size: 1.1, opacity: 0.8 },
|
{ name: 'DeepSeek', color: '#06b6d4', x: 13, y: 65, size: 1.1, opacity: 0.8 },
|
||||||
{ name: 'Grok', color: '#f43f5e', x: 52, y: 78, size: 1.0, opacity: 0.75 },
|
{ name: 'Grok', color: '#f43f5e', x: 52, y: 78, size: 1.0, opacity: 0.75 },
|
||||||
{ name: 'Ollama', color: '#9ca3af', x: 82, y: 25, size: 1.1, opacity: 0.8 },
|
{ name: 'Ollama', color: '#9ca3af', x: 84, y: 25, size: 1.1, opacity: 0.8 },
|
||||||
|
{ name: 'LM Studio', color: '#4338ca', x: 49, y: 36, size: 0.8, opacity: 0.9 },
|
||||||
{ name: 'Kimi', color: '#14b8a6', x: 30, y: 82, size: 0.85, opacity: 0.6 },
|
{ name: 'Kimi', color: '#14b8a6', x: 30, y: 82, size: 0.85, opacity: 0.6 },
|
||||||
{ name: 'GLM', color: '#f59e0b', x: 70, y: 72, size: 0.85, opacity: 0.55 },
|
{ name: 'GLM', color: '#f59e0b', x: 70, y: 72, size: 0.85, opacity: 0.55 },
|
||||||
{ name: 'LLaMA', color: '#60a5fa', x: 88, y: 70, size: 0.8, opacity: 0.45 },
|
{ name: 'LLaMA', color: '#60a5fa', x: 88, y: 70, size: 0.8, opacity: 0.45 },
|
||||||
|
|||||||
@@ -26,6 +26,7 @@ const SPA_ROUTES = [
|
|||||||
'docs/features/data-masking',
|
'docs/features/data-masking',
|
||||||
'docs/features/custom-instructions',
|
'docs/features/custom-instructions',
|
||||||
'docs/features/models',
|
'docs/features/models',
|
||||||
|
'docs/features/local-llms',
|
||||||
'docs/features/chrome-extension',
|
'docs/features/chrome-extension',
|
||||||
'docs/features/mcp-server',
|
'docs/features/mcp-server',
|
||||||
'docs/features/third-party-agent',
|
'docs/features/third-party-agent',
|
||||||
|
|||||||
Reference in New Issue
Block a user