docs: add comprehensive documentation for security.txt and canary.txt

Updated README with complete documentation for new features:

## Overview & Features
- Updated tagline to include security.txt and canary.txt
- Added 🔒 security.txt and 🐦 canary.txt to feature list

## Quick Start
- Added example showing how to enable security and canary files
- Shows resulting .well-known paths

## API Reference

### security section
- Complete TypeScript interface with all RFC 9116 fields
- Example configuration with common options
- Notes on mailto: prefix, auto-expiration, canonical URL

### canary section
- Full interface including CanaryStatement type
- Example with statements, verification, personnel check
- Frequency-based expiration table (daily→yearly)
- Links to CANARY_SPEC.md for full specification

## Caching
- Added security (24 hours) and canary (1 hour) cache defaults
- Note about frequent canary checking

## Advanced Usage
- Added custom template examples for both new files
- Shows proper typing and URL handling

Documentation now covers all 6 generated files with examples and best practices.
This commit is contained in:
Ryan Malloy 2025-11-03 08:21:26 -07:00
parent 2063d81e60
commit a686b22417

157
README.md
View File

@ -1,16 +1,18 @@
# @astrojs/discovery
> Comprehensive discovery integration for Astro - handles robots.txt, llms.txt, humans.txt, and sitemap generation
> Comprehensive discovery integration for Astro - handles robots.txt, llms.txt, humans.txt, security.txt, canary.txt, and sitemap generation
## Overview
This integration provides automatic generation of all standard discovery files for your Astro site, making it easily discoverable by search engines, LLMs, and humans.
This integration provides automatic generation of all standard discovery files for your Astro site, making it easily discoverable by search engines, LLMs, and humans, while providing security contact information and transparency mechanisms.
## Features
- 🤖 **robots.txt** - Dynamic generation with LLM bot support
- 🧠 **llms.txt** - AI assistant discovery and instructions
- 👥 **humans.txt** - Human-readable credits and tech stack
- 🔒 **security.txt** - RFC 9116 compliant security contact info
- 🐦 **canary.txt** - Warrant canary for transparency
- 🗺️ **sitemap.xml** - Automatic sitemap generation
- ⚡ **Dynamic URLs** - Adapts to your `site` config
- 🎯 **Smart Caching** - Optimized cache headers
@ -51,6 +53,29 @@ That's it! This will generate:
- `/humans.txt`
- `/sitemap-index.xml`
To enable security.txt and canary.txt, add their configurations:
```typescript
export default defineConfig({
site: 'https://example.com',
integrations: [
discovery({
security: {
contact: 'security@example.com',
},
canary: {
organization: 'Example Corp',
contact: 'canary@example.com',
}
})
]
});
```
This adds:
- `/.well-known/security.txt`
- `/.well-known/canary.txt`
### With Configuration
```typescript
@ -323,6 +348,108 @@ discovery({
})
```
##### `security`
Configuration for security.txt generation (RFC 9116).
**Type:**
```typescript
interface SecurityConfig {
enabled?: boolean;
contact: string | string[]; // Required: security contact (email or URL)
expires?: string | 'auto'; // Expiration date (default: 1 year)
encryption?: string | string[]; // PGP key URL(s)
acknowledgments?: string; // Hall of fame URL
preferredLanguages?: string[]; // Preferred languages (e.g., ['en', 'es'])
canonical?: string; // Canonical URL
policy?: string; // Security policy URL
hiring?: string; // Security jobs URL
}
```
**Example:**
```typescript
discovery({
security: {
contact: 'security@example.com',
expires: 'auto', // Auto-calculates 1 year from build
encryption: 'https://example.com/pgp-key.txt',
acknowledgments: 'https://example.com/security/hall-of-fame',
preferredLanguages: ['en', 'es'],
policy: 'https://example.com/security/policy'
}
})
```
**Notes:**
- Email contacts automatically get `mailto:` prefix
- `expires: 'auto'` sets expiration to 1 year from generation
- Generates at `/.well-known/security.txt` per RFC 9116
- Canonical URL defaults to correct .well-known location
##### `canary`
Configuration for warrant canary generation.
**Type:**
```typescript
interface CanaryConfig {
enabled?: boolean;
organization?: string; // Organization name
contact?: string; // Contact email
frequency?: 'daily' | 'weekly' | 'monthly' | 'quarterly' | 'yearly';
expires?: string | 'auto'; // Expiration (auto-calculated from frequency)
statements?: CanaryStatement[] | (() => CanaryStatement[]);
additionalStatement?: string; // Additional context
verification?: string; // PGP signature URL
previousCanary?: string; // Previous canary URL
blockchainProof?: { // Blockchain verification
network: string;
address: string;
txHash?: string;
timestamp?: string;
};
personnelStatement?: boolean; // Add duress check
}
interface CanaryStatement {
type: string;
description: string;
received: boolean;
}
```
**Example:**
```typescript
discovery({
canary: {
organization: 'Example Corp',
contact: 'canary@example.com',
frequency: 'monthly', // Auto-expires in 35 days
statements: [
{ type: 'nsl', description: 'National Security Letters', received: false },
{ type: 'gag', description: 'Gag orders', received: false }
],
additionalStatement: 'We are committed to transparency.',
verification: 'PGP Signature: https://example.com/canary.txt.asc',
personnelStatement: true // Adds duress check
}
})
```
**Frequency-based expiration:**
- `daily`: 2 days
- `weekly`: 10 days
- `monthly`: 35 days
- `quarterly`: 100 days
- `yearly`: 380 days
**Notes:**
- Only non-received statements appear in output
- Statements can be a function for dynamic generation
- Generates at `/.well-known/canary.txt`
- See [CANARY_SPEC.md](./CANARY_SPEC.md) for full specification
##### `sitemap`
Configuration passed to `@astrojs/sitemap`.
@ -361,9 +488,11 @@ Configure HTTP cache headers for discovery files.
**Type:**
```typescript
interface CachingConfig {
robots?: number; // seconds
robots?: number; // seconds
llms?: number;
humans?: number;
security?: number;
canary?: number;
sitemap?: number;
}
```
@ -371,10 +500,12 @@ interface CachingConfig {
**Default:**
```typescript
{
robots: 3600, // 1 hour
llms: 3600, // 1 hour
humans: 86400, // 24 hours
sitemap: 3600 // 1 hour
robots: 3600, // 1 hour
llms: 3600, // 1 hour
humans: 86400, // 24 hours
security: 86400, // 24 hours
canary: 3600, // 1 hour (check frequently!)
sitemap: 3600 // 1 hour
}
```
@ -399,6 +530,18 @@ Sitemap: ${siteURL}/sitemap-index.xml
# ${config.description}
Visit ${siteURL} for more information.
`,
security: (config, siteURL) => `
# Custom security.txt format
Contact: ${config.contact}
Expires: ${config.expires || new Date(Date.now() + 365*24*60*60*1000).toISOString()}
`,
canary: (config, siteURL) => `
# Custom canary format
Organization: ${config.organization}
Last-Updated: ${new Date().toISOString()}
`
}
})