Implemented two major new discovery file formats:
## security.txt (RFC 9116 - Industry Standard)
- Standardized security contact information
- Required fields: Contact, Expires, Canonical
- Optional fields: Encryption, Acknowledgments, Policy, Hiring, Languages
- Auto-expiration calculation (1 year by default)
- Proper mailto: prefix handling
- Located at /.well-known/security.txt
## canary.txt (NEW SPECIFICATION)
- First standardized format for warrant canaries
- Machine-readable transparency statements
- Auto-expiring based on update frequency
- Support for multiple statement types (NSL, FISA, gag orders, etc.)
- Optional blockchain proof
- Personnel duress statement
- See CANARY_SPEC.md for full specification
- Located at /.well-known/canary.txt
Changes:
- Added SecurityConfig and CanaryConfig type definitions
- Created generators for both formats with smart defaults
- Added API route handlers with virtual module config
- Updated integration to inject /.well-known/ routes
- Added to cache configuration (security: 24h, canary: 1h)
- Exported new types for TypeScript users
- Updated CLAUDE.md with feature priorities
- Created comprehensive CANARY_SPEC.md specification document
Testing: Both features verified in test project with full configuration.
All files generate correctly with proper formatting and validation.
The initial config-store approach failed because Astro's injected routes
run in isolated contexts during prerendering and don't have access to
global state set during astro:config:setup.
Solution: Created a Vite plugin that provides the configuration through
a virtual module (virtual:@astrojs/discovery/config) which routes can
import at build time.
Changes:
- Added Vite plugin in astro:config:setup hook
- Updated all route handlers to import from virtual module
- Changed version from date-based (2025.11.03) to semantic (1.0.0) per npm requirements
- Added @ts-ignore comments for virtual module imports
Testing: Verified in test project that all configuration now properly
passes through to generated files (robots.txt, llms.txt, humans.txt).
This commit introduces a comprehensive Astro integration that automatically
generates discovery files for websites:
Features:
- robots.txt with LLM bot support (Anthropic-AI, GPTBot, etc.)
- llms.txt for AI assistant context and instructions
- humans.txt for team credits and site information
- Automatic sitemap integration via @astrojs/sitemap
Technical Details:
- TypeScript implementation with full type safety
- Configurable HTTP caching headers
- Custom template support for all generated files
- Sensible defaults with extensive customization options
- Date-based versioning (2025.11.03)
Testing:
- 34 unit tests covering all generators
- Test coverage for robots.txt, llms.txt, and humans.txt
- Integration with Vitest
Documentation:
- Comprehensive README with examples
- API reference documentation
- Contributing guidelines
- Example configurations (minimal and full)
