docs: add comprehensive documentation for security.txt and canary.txt
Updated README with complete documentation for new features: ## Overview & Features - Updated tagline to include security.txt and canary.txt - Added 🔒 security.txt and 🐦 canary.txt to feature list ## Quick Start - Added example showing how to enable security and canary files - Shows resulting .well-known paths ## API Reference ### security section - Complete TypeScript interface with all RFC 9116 fields - Example configuration with common options - Notes on mailto: prefix, auto-expiration, canonical URL ### canary section - Full interface including CanaryStatement type - Example with statements, verification, personnel check - Frequency-based expiration table (daily→yearly) - Links to CANARY_SPEC.md for full specification ## Caching - Added security (24 hours) and canary (1 hour) cache defaults - Note about frequent canary checking ## Advanced Usage - Added custom template examples for both new files - Shows proper typing and URL handling Documentation now covers all 6 generated files with examples and best practices.
This commit is contained in:
parent
2063d81e60
commit
a686b22417
157
README.md
157
README.md
@ -1,16 +1,18 @@
|
||||
# @astrojs/discovery
|
||||
|
||||
> Comprehensive discovery integration for Astro - handles robots.txt, llms.txt, humans.txt, and sitemap generation
|
||||
> Comprehensive discovery integration for Astro - handles robots.txt, llms.txt, humans.txt, security.txt, canary.txt, and sitemap generation
|
||||
|
||||
## Overview
|
||||
|
||||
This integration provides automatic generation of all standard discovery files for your Astro site, making it easily discoverable by search engines, LLMs, and humans.
|
||||
This integration provides automatic generation of all standard discovery files for your Astro site, making it easily discoverable by search engines, LLMs, and humans, while providing security contact information and transparency mechanisms.
|
||||
|
||||
## Features
|
||||
|
||||
- 🤖 **robots.txt** - Dynamic generation with LLM bot support
|
||||
- 🧠 **llms.txt** - AI assistant discovery and instructions
|
||||
- 👥 **humans.txt** - Human-readable credits and tech stack
|
||||
- 🔒 **security.txt** - RFC 9116 compliant security contact info
|
||||
- 🐦 **canary.txt** - Warrant canary for transparency
|
||||
- 🗺️ **sitemap.xml** - Automatic sitemap generation
|
||||
- ⚡ **Dynamic URLs** - Adapts to your `site` config
|
||||
- 🎯 **Smart Caching** - Optimized cache headers
|
||||
@ -51,6 +53,29 @@ That's it! This will generate:
|
||||
- `/humans.txt`
|
||||
- `/sitemap-index.xml`
|
||||
|
||||
To enable security.txt and canary.txt, add their configurations:
|
||||
|
||||
```typescript
|
||||
export default defineConfig({
|
||||
site: 'https://example.com',
|
||||
integrations: [
|
||||
discovery({
|
||||
security: {
|
||||
contact: 'security@example.com',
|
||||
},
|
||||
canary: {
|
||||
organization: 'Example Corp',
|
||||
contact: 'canary@example.com',
|
||||
}
|
||||
})
|
||||
]
|
||||
});
|
||||
```
|
||||
|
||||
This adds:
|
||||
- `/.well-known/security.txt`
|
||||
- `/.well-known/canary.txt`
|
||||
|
||||
### With Configuration
|
||||
|
||||
```typescript
|
||||
@ -323,6 +348,108 @@ discovery({
|
||||
})
|
||||
```
|
||||
|
||||
##### `security`
|
||||
|
||||
Configuration for security.txt generation (RFC 9116).
|
||||
|
||||
**Type:**
|
||||
```typescript
|
||||
interface SecurityConfig {
|
||||
enabled?: boolean;
|
||||
contact: string | string[]; // Required: security contact (email or URL)
|
||||
expires?: string | 'auto'; // Expiration date (default: 1 year)
|
||||
encryption?: string | string[]; // PGP key URL(s)
|
||||
acknowledgments?: string; // Hall of fame URL
|
||||
preferredLanguages?: string[]; // Preferred languages (e.g., ['en', 'es'])
|
||||
canonical?: string; // Canonical URL
|
||||
policy?: string; // Security policy URL
|
||||
hiring?: string; // Security jobs URL
|
||||
}
|
||||
```
|
||||
|
||||
**Example:**
|
||||
```typescript
|
||||
discovery({
|
||||
security: {
|
||||
contact: 'security@example.com',
|
||||
expires: 'auto', // Auto-calculates 1 year from build
|
||||
encryption: 'https://example.com/pgp-key.txt',
|
||||
acknowledgments: 'https://example.com/security/hall-of-fame',
|
||||
preferredLanguages: ['en', 'es'],
|
||||
policy: 'https://example.com/security/policy'
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
**Notes:**
|
||||
- Email contacts automatically get `mailto:` prefix
|
||||
- `expires: 'auto'` sets expiration to 1 year from generation
|
||||
- Generates at `/.well-known/security.txt` per RFC 9116
|
||||
- Canonical URL defaults to correct .well-known location
|
||||
|
||||
##### `canary`
|
||||
|
||||
Configuration for warrant canary generation.
|
||||
|
||||
**Type:**
|
||||
```typescript
|
||||
interface CanaryConfig {
|
||||
enabled?: boolean;
|
||||
organization?: string; // Organization name
|
||||
contact?: string; // Contact email
|
||||
frequency?: 'daily' | 'weekly' | 'monthly' | 'quarterly' | 'yearly';
|
||||
expires?: string | 'auto'; // Expiration (auto-calculated from frequency)
|
||||
statements?: CanaryStatement[] | (() => CanaryStatement[]);
|
||||
additionalStatement?: string; // Additional context
|
||||
verification?: string; // PGP signature URL
|
||||
previousCanary?: string; // Previous canary URL
|
||||
blockchainProof?: { // Blockchain verification
|
||||
network: string;
|
||||
address: string;
|
||||
txHash?: string;
|
||||
timestamp?: string;
|
||||
};
|
||||
personnelStatement?: boolean; // Add duress check
|
||||
}
|
||||
|
||||
interface CanaryStatement {
|
||||
type: string;
|
||||
description: string;
|
||||
received: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
**Example:**
|
||||
```typescript
|
||||
discovery({
|
||||
canary: {
|
||||
organization: 'Example Corp',
|
||||
contact: 'canary@example.com',
|
||||
frequency: 'monthly', // Auto-expires in 35 days
|
||||
statements: [
|
||||
{ type: 'nsl', description: 'National Security Letters', received: false },
|
||||
{ type: 'gag', description: 'Gag orders', received: false }
|
||||
],
|
||||
additionalStatement: 'We are committed to transparency.',
|
||||
verification: 'PGP Signature: https://example.com/canary.txt.asc',
|
||||
personnelStatement: true // Adds duress check
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
**Frequency-based expiration:**
|
||||
- `daily`: 2 days
|
||||
- `weekly`: 10 days
|
||||
- `monthly`: 35 days
|
||||
- `quarterly`: 100 days
|
||||
- `yearly`: 380 days
|
||||
|
||||
**Notes:**
|
||||
- Only non-received statements appear in output
|
||||
- Statements can be a function for dynamic generation
|
||||
- Generates at `/.well-known/canary.txt`
|
||||
- See [CANARY_SPEC.md](./CANARY_SPEC.md) for full specification
|
||||
|
||||
##### `sitemap`
|
||||
|
||||
Configuration passed to `@astrojs/sitemap`.
|
||||
@ -361,9 +488,11 @@ Configure HTTP cache headers for discovery files.
|
||||
**Type:**
|
||||
```typescript
|
||||
interface CachingConfig {
|
||||
robots?: number; // seconds
|
||||
robots?: number; // seconds
|
||||
llms?: number;
|
||||
humans?: number;
|
||||
security?: number;
|
||||
canary?: number;
|
||||
sitemap?: number;
|
||||
}
|
||||
```
|
||||
@ -371,10 +500,12 @@ interface CachingConfig {
|
||||
**Default:**
|
||||
```typescript
|
||||
{
|
||||
robots: 3600, // 1 hour
|
||||
llms: 3600, // 1 hour
|
||||
humans: 86400, // 24 hours
|
||||
sitemap: 3600 // 1 hour
|
||||
robots: 3600, // 1 hour
|
||||
llms: 3600, // 1 hour
|
||||
humans: 86400, // 24 hours
|
||||
security: 86400, // 24 hours
|
||||
canary: 3600, // 1 hour (check frequently!)
|
||||
sitemap: 3600 // 1 hour
|
||||
}
|
||||
```
|
||||
|
||||
@ -399,6 +530,18 @@ Sitemap: ${siteURL}/sitemap-index.xml
|
||||
# ${config.description}
|
||||
|
||||
Visit ${siteURL} for more information.
|
||||
`,
|
||||
|
||||
security: (config, siteURL) => `
|
||||
# Custom security.txt format
|
||||
Contact: ${config.contact}
|
||||
Expires: ${config.expires || new Date(Date.now() + 365*24*60*60*1000).toISOString()}
|
||||
`,
|
||||
|
||||
canary: (config, siteURL) => `
|
||||
# Custom canary format
|
||||
Organization: ${config.organization}
|
||||
Last-Updated: ${new Date().toISOString()}
|
||||
`
|
||||
}
|
||||
})
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user