Ryan Malloy
d25dde4627
This commit introduces a comprehensive Astro integration that automatically generates discovery files for websites: Features: - robots.txt with LLM bot support (Anthropic-AI, GPTBot, etc.) - llms.txt for AI assistant context and instructions - humans.txt for team credits and site information - Automatic sitemap integration via @astrojs/sitemap Technical Details: - TypeScript implementation with full type safety - Configurable HTTP caching headers - Custom template support for all generated files - Sensible defaults with extensive customization options - Date-based versioning (2025.11.03) Testing: - 34 unit tests covering all generators - Test coverage for robots.txt, llms.txt, and humans.txt - Integration with Vitest Documentation: - Comprehensive README with examples - API reference documentation - Contributing guidelines - Example configurations (minimal and full)
82 lines
2.0 KiB
Markdown
82 lines
2.0 KiB
Markdown
# Changelog
|
|
|
|
All notable changes to @astrojs/discovery will be documented in this file.
|
|
|
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
and this project uses date-based versioning (YYYY.MM.DD).
|
|
|
|
## [2025.11.03] - 2025-11-03
|
|
|
|
### Added
|
|
- Initial release of @astrojs/discovery
|
|
- Automatic robots.txt generation with LLM bot support
|
|
- Automatic llms.txt generation for AI assistant context
|
|
- Automatic humans.txt generation for team credits
|
|
- Integration with @astrojs/sitemap for sitemap-index.xml
|
|
- Configurable HTTP caching headers
|
|
- Custom template support for all generated files
|
|
- TypeScript type definitions
|
|
- Comprehensive configuration options
|
|
- Example configurations (minimal and full)
|
|
|
|
### Features
|
|
- **robots.txt**
|
|
- Default allow-all policy
|
|
- LLM-specific bot rules (Anthropic-AI, GPTBot, etc.)
|
|
- Custom agent configurations
|
|
- Crawl delay settings
|
|
- Custom rules support
|
|
|
|
- **llms.txt**
|
|
- Site description and key features
|
|
- Important pages listing
|
|
- AI assistant instructions
|
|
- API endpoint documentation
|
|
- Technology stack information
|
|
- Brand voice guidelines
|
|
- Custom sections
|
|
|
|
- **humans.txt**
|
|
- Team member information
|
|
- Thanks/credits section
|
|
- Site technical information
|
|
- Project story
|
|
- Fun facts
|
|
- Development philosophy
|
|
- Custom sections
|
|
|
|
- **Configuration**
|
|
- Sensible defaults
|
|
- Full customization options
|
|
- Environment-based toggles
|
|
- Dynamic content support
|
|
- Cache control configuration
|
|
|
|
### Documentation
|
|
- Comprehensive README with examples
|
|
- API reference documentation
|
|
- Contributing guidelines
|
|
- Example configurations
|
|
- Integration guides
|
|
|
|
## Future Enhancements
|
|
|
|
### Planned Features
|
|
- security.txt support (RFC 9116)
|
|
- ads.txt support for advertising
|
|
- manifest.json for PWA
|
|
- RSS feed integration
|
|
- OpenGraph tags injection
|
|
- Structured data (JSON-LD)
|
|
- Analytics discovery
|
|
- i18n support for multi-language sites
|
|
|
|
### Testing
|
|
- Unit tests for generators
|
|
- Integration tests
|
|
- E2E tests with real Astro projects
|
|
|
|
---
|
|
|
|
For more information, see [README.md](README.md)
|