Ryan Malloy
6120506e91
Adds revolutionary features for MCP client identification and browser automation: MCP Client Debug System: - Floating pill toolbar with client identification and session info - Theme system with 5 built-in themes (minimal, corporate, hacker, glass, high-contrast) - Custom theme creation API with CSS variable overrides - Cross-site validation ensuring toolbar persists across navigation - Session-based injection with persistence across page loads Voice Collaboration (Prototype): - Web Speech API integration for conversational browser automation - Bidirectional voice communication between AI and user - Real-time voice guidance during automation tasks - Documented architecture and future development roadmap Code Injection Enhancements: - Model collaboration API for notify, prompt, and inspector functions - Auto-injection and persistence options - Toolbar integration with code injection system Documentation: - Comprehensive technical achievement documentation - Voice collaboration architecture and implementation guide - Theme system integration documentation - Tool annotation templates for consistency This represents a major advancement in browser automation UX, enabling unprecedented visibility and interaction patterns for MCP clients.
2.0 KiB
2.0 KiB
MCP Tool Annotation Template
For inclusion in tool descriptions/schemas:
// Add this annotation to any Playwright MCP tool description:
const toolDescription = `
Navigate to a URL and interact with web pages.
🤖 COLLABORATION API AVAILABLE:
Models can communicate directly with users through JavaScript functions:
• mcpNotify.info('message') - Send info to user
• mcpNotify.success('completed!') - Show success
• mcpNotify.warning('be careful') - Display warnings
• mcpNotify.error('something failed') - Show errors
• mcpNotify.loading('processing...') - Persistent status
• await mcpPrompt('Shall I proceed?') - Get user confirmation
• mcpInspector.start('Click the login button', callback) - Interactive element selection
When elements are ambiguous or actions need confirmation, use these functions
to collaborate with the user for better automation results.
Full API: See MODEL-COLLABORATION-API.md
`;
Short version for tool schemas:
const shortAnnotation = `
🤖 MODELS: Use mcpNotify.info('message'), mcpPrompt('question?'), and
mcpInspector.start('click element', callback) for user collaboration.
`;
For browser_evaluate tool specifically:
const evaluateDescription = `
Evaluate JavaScript on the page.
🤖 COLLABORATION FUNCTIONS AVAILABLE:
After running this tool, models can use JavaScript to communicate with users:
- mcpMessage(), mcpNotify.*() for messages
- mcpPrompt() for confirmations
- mcpInspector.start() for element selection
Example: await page.evaluate(() => mcpNotify.success('Task completed!'));
`;
Implementation Strategy:
- Add to existing tool descriptions - Append the short annotation
- Include in documentation - Reference the full MODEL-COLLABORATION-API.md
- Tool-specific notes - Add context-relevant collaboration hints
- Examples in schemas - Show practical usage patterns
This ensures models discover and use the collaboration features naturally while using the MCP tools.