- Change package name from @astrojs/discovery to @supsys/discovery
- Update repository URL to git.supported.systems
- Update CHANGELOG references
- Rebuild and verify all tests pass (89/89)
Complete API reference for all configuration options
- 11 reference pages covering entire API surface
- TypeScript type definitions
- Default values and examples
- RFC compliance documentation
Add comprehensive problem-oriented how-to guides following Diátaxis framework:
- Block specific bots from crawling the site
- Customize LLM instructions for AI assistants
- Add team members to humans.txt
- Filter sitemap pages
- Configure cache headers for discovery files
- Environment-specific configuration
- Integration with Astro content collections
- Custom templates for discovery files
- ActivityPub/Fediverse integration via WebFinger
Each guide provides:
- Clear prerequisites
- Step-by-step solutions
- Multiple approaches/variations
- Expected outcomes
- Alternative approaches
- Common issues and troubleshooting
Total: 9 guides, 6,677 words
Completed comprehensive, learning-oriented tutorial content following Diátaxis framework:
Getting Started pages:
- first-steps.md: Hands-on exploration of generated files with step-by-step verification
- installation.md and quick-start.md: Already completed with clear installation paths
Tutorial pages:
- basic-setup.md: Progressive configuration from minimal to complete setup
- configure-robots.md: Bot access control with practical examples
- setup-llms.md: AI assistant context with instructions and API documentation
- create-humans.md: Team credits, tech stack, and project story
- security-canary.md: RFC 9116 security.txt and warrant canary implementation
- webfinger.md: Federated discovery for ActivityPub and OpenID Connect
All tutorials follow Diátaxis principles:
- Step-by-step instructions that work for everyone
- Concrete code examples with expected output
- Build and verify commands after each step
- Focus on getting learners started with confidence
- Minimal explanation, maximum hands-on practice
- Progressive complexity building on previous steps
Documentation verified with successful docs build.
Add complete Reference documentation for all @astrojs/discovery configuration
and API surfaces following Diátaxis information-oriented principles:
Reference pages completed:
- configuration.md: Main configuration interface with all options
- robots.md: Complete robots.txt configuration reference
- llms.md: Complete llms.txt configuration reference
- humans.md: Complete humans.txt configuration reference
- security.md: RFC 9116 compliant security.txt reference
- canary.md: Warrant canary configuration reference
- webfinger.md: RFC 7033 WebFinger configuration reference
- sitemap.md: Sitemap configuration (via @astrojs/sitemap)
- cache.md: HTTP caching configuration reference
- typescript.md: Complete TypeScript type definitions
- api.md: Programmatic API reference
Documentation characteristics:
- Information-oriented: Describes the machinery accurately
- Austere and to the point: No tutorials, just facts
- Structure around code: Follows TypeScript interfaces
- Consistent structure: All pages follow same format
- Do nothing but describe: Pure reference material
- Provide examples: Code examples for all options
- Be accurate: Matches actual implementation
Each page includes:
- TypeScript interface definitions
- Property descriptions with types and defaults
- Code examples for all configuration options
- Common patterns and use cases
- Output location and format information
- Best practices where applicable
All 11 reference pages now provide complete, accurate API documentation
suitable for experienced users who need to look up specific details.
- Created comprehensive Diátaxis-based documentation
- 53 pages across Tutorials, How-to, Reference, and Explanation
- Custom branding with @astrojs/discovery logo
- Pagefind search integration
- Ready for content population
Updated README with complete WebFinger section:
## Overview & Features
- Updated tagline to include WebFinger
- Added 🔍 WebFinger to feature list with use cases
## API Reference - webfinger section
### Complete TypeScript Interfaces
- WebFingerConfig with enabled, resources, collections
- WebFingerResource (JRD structure)
- WebFingerLink (rel, href, type, titles, properties)
### Static Resources Example
- Shows simple acct: URI configuration
- ActivityPub/Mastodon integration example
- Profile page and avatar links
### Content Collection Example
- Team members as discoverable resources
- Template URI patterns: acct:{slug}@example.com
- Builder functions for links and properties
- Schema.org property integration
### Common Use Cases
- ActivityPub/Mastodon federation
- OpenID Connect issuer discovery
- Team profile discovery
- Blog author linking
### Query Format Documentation
- Required resource parameter
- Optional rel filtering
- Example queries
### Technical Notes
- Dynamic route (not prerendered)
- CORS enabled per RFC 7033
- Media type: application/jrd+json
- Template variable reference
## Caching Section
- Added webfinger: 3600 (1 hour) to defaults table
Documentation now covers all 7 discovery mechanisms with examples and best practices.
Added complete test coverage for WebFinger JRD generation:
## Static Resources Tests (11 tests)
- Unknown resource returns null
- Basic JRD generation
- Subject defaults to resource when not provided
- Aliases support
- Properties support (URI-based names)
- Links support with rel, href, type
- Rel filtering (single and multiple)
- Link titles with language tags
- Complete JRD with all fields
## Protocol Support Tests
- HTTP/HTTPS resource URIs
- ActivityPub profile discovery (self rel, application/activity+json)
- OpenID Connect issuer discovery
## Edge Cases (4 tests)
- Resource with no links
- Empty aliases array omitted
- Empty properties object omitted
- All links filtered out by rel
Test suite now at 89 tests (up from 72), all passing.
Coverage includes RFC 7033 compliance, common use cases (Mastodon/ActivityPub,
OpenID), and edge cases.
Updated README with complete documentation for new features:
## Overview & Features
- Updated tagline to include security.txt and canary.txt
- Added 🔒 security.txt and 🐦 canary.txt to feature list
## Quick Start
- Added example showing how to enable security and canary files
- Shows resulting .well-known paths
## API Reference
### security section
- Complete TypeScript interface with all RFC 9116 fields
- Example configuration with common options
- Notes on mailto: prefix, auto-expiration, canonical URL
### canary section
- Full interface including CanaryStatement type
- Example with statements, verification, personnel check
- Frequency-based expiration table (daily→yearly)
- Links to CANARY_SPEC.md for full specification
## Caching
- Added security (24 hours) and canary (1 hour) cache defaults
- Note about frequent canary checking
## Advanced Usage
- Added custom template examples for both new files
- Shows proper typing and URL handling
Documentation now covers all 6 generated files with examples and best practices.
Implemented two major new discovery file formats:
## security.txt (RFC 9116 - Industry Standard)
- Standardized security contact information
- Required fields: Contact, Expires, Canonical
- Optional fields: Encryption, Acknowledgments, Policy, Hiring, Languages
- Auto-expiration calculation (1 year by default)
- Proper mailto: prefix handling
- Located at /.well-known/security.txt
## canary.txt (NEW SPECIFICATION)
- First standardized format for warrant canaries
- Machine-readable transparency statements
- Auto-expiring based on update frequency
- Support for multiple statement types (NSL, FISA, gag orders, etc.)
- Optional blockchain proof
- Personnel duress statement
- See CANARY_SPEC.md for full specification
- Located at /.well-known/canary.txt
Changes:
- Added SecurityConfig and CanaryConfig type definitions
- Created generators for both formats with smart defaults
- Added API route handlers with virtual module config
- Updated integration to inject /.well-known/ routes
- Added to cache configuration (security: 24h, canary: 1h)
- Exported new types for TypeScript users
- Updated CLAUDE.md with feature priorities
- Created comprehensive CANARY_SPEC.md specification document
Testing: Both features verified in test project with full configuration.
All files generate correctly with proper formatting and validation.
The initial config-store approach failed because Astro's injected routes
run in isolated contexts during prerendering and don't have access to
global state set during astro:config:setup.
Solution: Created a Vite plugin that provides the configuration through
a virtual module (virtual:@astrojs/discovery/config) which routes can
import at build time.
Changes:
- Added Vite plugin in astro:config:setup hook
- Updated all route handlers to import from virtual module
- Changed version from date-based (2025.11.03) to semantic (1.0.0) per npm requirements
- Added @ts-ignore comments for virtual module imports
Testing: Verified in test project that all configuration now properly
passes through to generated files (robots.txt, llms.txt, humans.txt).
This commit introduces a comprehensive Astro integration that automatically
generates discovery files for websites:
Features:
- robots.txt with LLM bot support (Anthropic-AI, GPTBot, etc.)
- llms.txt for AI assistant context and instructions
- humans.txt for team credits and site information
- Automatic sitemap integration via @astrojs/sitemap
Technical Details:
- TypeScript implementation with full type safety
- Configurable HTTP caching headers
- Custom template support for all generated files
- Sensible defaults with extensive customization options
- Date-based versioning (2025.11.03)
Testing:
- 34 unit tests covering all generators
- Test coverage for robots.txt, llms.txt, and humans.txt
- Integration with Vitest
Documentation:
- Comprehensive README with examples
- API reference documentation
- Contributing guidelines
- Example configurations (minimal and full)
