feat: add security.txt (RFC 9116) and canary.txt support
Implemented two major new discovery file formats: ## security.txt (RFC 9116 - Industry Standard) - Standardized security contact information - Required fields: Contact, Expires, Canonical - Optional fields: Encryption, Acknowledgments, Policy, Hiring, Languages - Auto-expiration calculation (1 year by default) - Proper mailto: prefix handling - Located at /.well-known/security.txt ## canary.txt (NEW SPECIFICATION) - First standardized format for warrant canaries - Machine-readable transparency statements - Auto-expiring based on update frequency - Support for multiple statement types (NSL, FISA, gag orders, etc.) - Optional blockchain proof - Personnel duress statement - See CANARY_SPEC.md for full specification - Located at /.well-known/canary.txt Changes: - Added SecurityConfig and CanaryConfig type definitions - Created generators for both formats with smart defaults - Added API route handlers with virtual module config - Updated integration to inject /.well-known/ routes - Added to cache configuration (security: 24h, canary: 1h) - Exported new types for TypeScript users - Updated CLAUDE.md with feature priorities - Created comprehensive CANARY_SPEC.md specification document Testing: Both features verified in test project with full configuration. All files generate correctly with proper formatting and validation.
This commit is contained in:
parent
c7b47bba5c
commit
6de34f55a9
272
CANARY_SPEC.md
Normal file
272
CANARY_SPEC.md
Normal file
@ -0,0 +1,272 @@
|
|||||||
|
# canary.txt Specification
|
||||||
|
|
||||||
|
**Version:** 1.0.0-draft
|
||||||
|
**Status:** Proposal
|
||||||
|
**Author:** Ryan Malloy
|
||||||
|
**Date:** 2025-11-03
|
||||||
|
|
||||||
|
## Abstract
|
||||||
|
|
||||||
|
This document defines `canary.txt`, a standard format for warrant canaries - public statements that organizations publish to indicate they have NOT received secret government requests (national security letters, gag orders, etc.).
|
||||||
|
|
||||||
|
## Motivation
|
||||||
|
|
||||||
|
Warrant canaries currently exist in various ad-hoc formats across websites, making them:
|
||||||
|
- Hard to discover
|
||||||
|
- Difficult to verify programmatically
|
||||||
|
- Inconsistent in format
|
||||||
|
- Challenging to monitor at scale
|
||||||
|
|
||||||
|
A standardized format enables:
|
||||||
|
- Automated monitoring services
|
||||||
|
- Browser extensions that alert users
|
||||||
|
- Transparency dashboards
|
||||||
|
- Cryptographic verification
|
||||||
|
- Historical tracking
|
||||||
|
|
||||||
|
## Specification
|
||||||
|
|
||||||
|
### Location
|
||||||
|
|
||||||
|
The canary file MUST be located at one of:
|
||||||
|
- `/.well-known/canary.txt` (RECOMMENDED)
|
||||||
|
- `/canary.txt` (ACCEPTABLE)
|
||||||
|
|
||||||
|
### Format
|
||||||
|
|
||||||
|
The file MUST be plain text (text/plain; charset=utf-8).
|
||||||
|
|
||||||
|
### Required Fields
|
||||||
|
|
||||||
|
```
|
||||||
|
Canonical-URL: https://example.com/.well-known/canary.txt
|
||||||
|
Issued: 2025-11-03T00:00:00Z
|
||||||
|
Expires: 2026-11-03T00:00:00Z
|
||||||
|
```
|
||||||
|
|
||||||
|
**Canonical-URL:** The authoritative URL for this canary
|
||||||
|
**Issued:** ISO 8601 timestamp when this canary was published
|
||||||
|
**Expires:** ISO 8601 timestamp when this canary expires
|
||||||
|
|
||||||
|
### Statement Section
|
||||||
|
|
||||||
|
The canary MUST include a clear statement:
|
||||||
|
|
||||||
|
```
|
||||||
|
Statement: As of 2025-11-03, Example Inc has NOT received:
|
||||||
|
- National Security Letters (NSLs)
|
||||||
|
- FISA court orders
|
||||||
|
- Gag orders preventing disclosure
|
||||||
|
- Secret government requests for user data
|
||||||
|
- Requests to install surveillance capabilities
|
||||||
|
```
|
||||||
|
|
||||||
|
### Optional Fields
|
||||||
|
|
||||||
|
```
|
||||||
|
Organization: Example Inc
|
||||||
|
Contact: legal@example.com
|
||||||
|
Verification: https://example.com/canary-verification.asc
|
||||||
|
Frequency: monthly
|
||||||
|
Previous-Canary: https://example.com/.well-known/canary-2025-10.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
**Organization:** Legal entity name
|
||||||
|
**Contact:** Email for canary-related inquiries
|
||||||
|
**Verification:** URL to PGP signature or blockchain proof
|
||||||
|
**Frequency:** Update frequency (daily, weekly, monthly, quarterly)
|
||||||
|
**Previous-Canary:** Link to previous canary (for continuity)
|
||||||
|
|
||||||
|
### PGP Signature (Recommended)
|
||||||
|
|
||||||
|
Canaries SHOULD be PGP-signed for authenticity:
|
||||||
|
|
||||||
|
```
|
||||||
|
-----BEGIN PGP SIGNED MESSAGE-----
|
||||||
|
Hash: SHA512
|
||||||
|
|
||||||
|
Canonical-URL: https://example.com/.well-known/canary.txt
|
||||||
|
Issued: 2025-11-03T00:00:00Z
|
||||||
|
Expires: 2026-11-03T00:00:00Z
|
||||||
|
|
||||||
|
Statement: As of 2025-11-03, Example Inc has NOT received:
|
||||||
|
- National Security Letters (NSLs)
|
||||||
|
- FISA court orders
|
||||||
|
...
|
||||||
|
|
||||||
|
-----BEGIN PGP SIGNATURE-----
|
||||||
|
...
|
||||||
|
-----END PGP SIGNATURE-----
|
||||||
|
```
|
||||||
|
|
||||||
|
### Blockchain Proof (Optional)
|
||||||
|
|
||||||
|
For additional verification, include:
|
||||||
|
|
||||||
|
```
|
||||||
|
Blockchain-Proof: bitcoin:1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa:blockhash
|
||||||
|
Blockchain-Timestamp: 2025-11-03T00:00:00Z
|
||||||
|
```
|
||||||
|
|
||||||
|
## Example: Minimal Canary
|
||||||
|
|
||||||
|
```
|
||||||
|
Canonical-URL: https://example.com/.well-known/canary.txt
|
||||||
|
Issued: 2025-11-03T00:00:00Z
|
||||||
|
Expires: 2026-02-03T00:00:00Z
|
||||||
|
Organization: Example Inc
|
||||||
|
Contact: transparency@example.com
|
||||||
|
|
||||||
|
Statement: As of 2025-11-03, Example Inc has NOT received any:
|
||||||
|
- National Security Letters
|
||||||
|
- FISA court orders
|
||||||
|
- Gag orders
|
||||||
|
- Secret government requests
|
||||||
|
|
||||||
|
This canary will be updated quarterly. Absence of an update
|
||||||
|
should be considered significant.
|
||||||
|
```
|
||||||
|
|
||||||
|
## Example: Maximum Security Canary
|
||||||
|
|
||||||
|
```
|
||||||
|
-----BEGIN PGP SIGNED MESSAGE-----
|
||||||
|
Hash: SHA512
|
||||||
|
|
||||||
|
Canonical-URL: https://example.com/.well-known/canary.txt
|
||||||
|
Issued: 2025-11-03T00:00:00Z
|
||||||
|
Expires: 2025-12-03T00:00:00Z
|
||||||
|
Organization: Privacy-First Corp
|
||||||
|
Contact: canary@privacyfirst.example
|
||||||
|
Verification: https://privacyfirst.example/pgp-public.asc
|
||||||
|
Frequency: monthly
|
||||||
|
Previous-Canary: https://privacyfirst.example/.well-known/canary-2025-10.txt
|
||||||
|
Blockchain-Proof: bitcoin:1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa:00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048
|
||||||
|
Blockchain-Timestamp: 2025-11-03T00:05:23Z
|
||||||
|
|
||||||
|
Statement: As of 2025-11-03, Privacy-First Corp has NOT:
|
||||||
|
- Received any National Security Letters (NSLs)
|
||||||
|
- Received any FISA court orders
|
||||||
|
- Received any gag orders preventing disclosure
|
||||||
|
- Been compelled to modify our software or services
|
||||||
|
- Been compelled to install surveillance capabilities
|
||||||
|
- Received requests to weaken encryption
|
||||||
|
- Been prohibited from updating this canary
|
||||||
|
|
||||||
|
This statement is updated monthly on the first business day.
|
||||||
|
Any absence of an update within 35 days should be considered
|
||||||
|
a removal of this canary.
|
||||||
|
|
||||||
|
Key Personnel Statement: All key personnel with access to
|
||||||
|
infrastructure remain free and under no duress.
|
||||||
|
|
||||||
|
-----BEGIN PGP SIGNATURE-----
|
||||||
|
iQIzBAEBCgAdFiEE...
|
||||||
|
-----END PGP SIGNATURE-----
|
||||||
|
```
|
||||||
|
|
||||||
|
## Machine-Readable Format (.well-known/canary.json)
|
||||||
|
|
||||||
|
For programmatic consumption:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"version": "1.0",
|
||||||
|
"canonical_url": "https://example.com/.well-known/canary.txt",
|
||||||
|
"issued": "2025-11-03T00:00:00Z",
|
||||||
|
"expires": "2026-11-03T00:00:00Z",
|
||||||
|
"organization": "Example Inc",
|
||||||
|
"contact": "transparency@example.com",
|
||||||
|
"frequency": "monthly",
|
||||||
|
"statements": [
|
||||||
|
{
|
||||||
|
"type": "nsl",
|
||||||
|
"received": false,
|
||||||
|
"description": "National Security Letters"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "fisa",
|
||||||
|
"received": false,
|
||||||
|
"description": "FISA court orders"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "gag",
|
||||||
|
"received": false,
|
||||||
|
"description": "Gag orders"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"verification": {
|
||||||
|
"pgp_signature_url": "https://example.com/canary-verification.asc",
|
||||||
|
"pgp_fingerprint": "1234 5678 9ABC DEF0 1234 5678 9ABC DEF0 1234 5678",
|
||||||
|
"blockchain": {
|
||||||
|
"network": "bitcoin",
|
||||||
|
"address": "1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa",
|
||||||
|
"tx_hash": "blockhash"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"previous_canary": "https://example.com/.well-known/canary-2025-10.txt"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Monitoring & Verification
|
||||||
|
|
||||||
|
### Automated Monitoring
|
||||||
|
|
||||||
|
Services can:
|
||||||
|
1. Fetch canary regularly
|
||||||
|
2. Verify PGP signature
|
||||||
|
3. Check expiration dates
|
||||||
|
4. Alert on missing/changed canaries
|
||||||
|
5. Track historical versions
|
||||||
|
|
||||||
|
### Browser Extension Concept
|
||||||
|
|
||||||
|
A browser extension could:
|
||||||
|
- Display canary status icon
|
||||||
|
- Alert when canary expires/changes
|
||||||
|
- Show canary history
|
||||||
|
- Verify signatures
|
||||||
|
|
||||||
|
## Legal Considerations
|
||||||
|
|
||||||
|
**IMPORTANT:** Consult legal counsel before implementing a warrant canary.
|
||||||
|
- Effectiveness varies by jurisdiction
|
||||||
|
- May not be legally tested in all countries
|
||||||
|
- Should not be sole transparency mechanism
|
||||||
|
- Consider comprehensive transparency reports
|
||||||
|
|
||||||
|
## Security Properties
|
||||||
|
|
||||||
|
1. **Passive Statement:** Says what has NOT happened
|
||||||
|
2. **Absence is Signal:** Missing canary = possible receipt of order
|
||||||
|
3. **Cryptographic Proof:** PGP prevents forgery
|
||||||
|
4. **Blockchain Anchoring:** Proves existence at point in time
|
||||||
|
5. **Continuous Verification:** Automated monitoring detects changes
|
||||||
|
|
||||||
|
## Limitations
|
||||||
|
|
||||||
|
- Legal effectiveness uncertain in many jurisdictions
|
||||||
|
- Sophisticated adversaries may compel deceptive updates
|
||||||
|
- Does not guarantee absence of other surveillance
|
||||||
|
- Should complement, not replace, other transparency measures
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- RFC 8615: Well-Known URIs
|
||||||
|
- RFC 9116: security.txt
|
||||||
|
- EFF: Warrant Canary FAQ
|
||||||
|
- Warrant Canary Watch
|
||||||
|
|
||||||
|
## Contributing
|
||||||
|
|
||||||
|
This is a draft specification. Feedback welcome at:
|
||||||
|
- GitHub: [repository URL]
|
||||||
|
- Email: ryan@supported.systems
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
This specification is released under CC0 1.0 Universal (Public Domain).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Status:** This is a PROPOSED standard, not yet ratified by any standards body.
|
||||||
164
src/generators/canary.ts
Normal file
164
src/generators/canary.ts
Normal file
@ -0,0 +1,164 @@
|
|||||||
|
import type { CanaryConfig, CanaryStatement } from '../types.js';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Default canary statements (what has NOT been received)
|
||||||
|
*/
|
||||||
|
const DEFAULT_STATEMENTS: CanaryStatement[] = [
|
||||||
|
{
|
||||||
|
type: 'nsl',
|
||||||
|
description: 'National Security Letters (NSLs)',
|
||||||
|
received: false,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
type: 'fisa',
|
||||||
|
description: 'FISA court orders',
|
||||||
|
received: false,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
type: 'gag',
|
||||||
|
description: 'Gag orders preventing disclosure',
|
||||||
|
received: false,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
type: 'surveillance',
|
||||||
|
description: 'Secret government requests for user data',
|
||||||
|
received: false,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
type: 'backdoor',
|
||||||
|
description: 'Requests to install surveillance capabilities',
|
||||||
|
received: false,
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Calculate expiration date based on frequency
|
||||||
|
*/
|
||||||
|
function calculateExpiration(frequency: string): string {
|
||||||
|
const now = new Date();
|
||||||
|
const expirationMap: Record<string, number> = {
|
||||||
|
daily: 2, // 2 days
|
||||||
|
weekly: 10, // 10 days
|
||||||
|
monthly: 35, // 35 days
|
||||||
|
quarterly: 100, // ~100 days
|
||||||
|
yearly: 380, // 380 days
|
||||||
|
};
|
||||||
|
|
||||||
|
const days = expirationMap[frequency] || 35;
|
||||||
|
const expiration = new Date(now.getTime() + days * 24 * 60 * 60 * 1000);
|
||||||
|
return expiration.toISOString();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Generate canary.txt content
|
||||||
|
*
|
||||||
|
* @param config - Canary configuration
|
||||||
|
* @param siteURL - Site base URL
|
||||||
|
* @returns Generated canary.txt content
|
||||||
|
*/
|
||||||
|
export function generateCanaryTxt(
|
||||||
|
config: CanaryConfig,
|
||||||
|
siteURL: URL
|
||||||
|
): string {
|
||||||
|
const lines: string[] = [];
|
||||||
|
const today = new Date().toISOString().split('T')[0];
|
||||||
|
|
||||||
|
// Canonical URL
|
||||||
|
const canonical = new URL('.well-known/canary.txt', siteURL).href;
|
||||||
|
lines.push(`Canonical-URL: ${canonical}`);
|
||||||
|
|
||||||
|
// Issued date (today)
|
||||||
|
lines.push(`Issued: ${new Date().toISOString()}`);
|
||||||
|
|
||||||
|
// Expires date
|
||||||
|
const expires = config.expires === 'auto'
|
||||||
|
? calculateExpiration(config.frequency || 'monthly')
|
||||||
|
: config.expires || calculateExpiration('monthly');
|
||||||
|
lines.push(`Expires: ${expires}`);
|
||||||
|
|
||||||
|
// Organization (optional)
|
||||||
|
if (config.organization) {
|
||||||
|
lines.push(`Organization: ${config.organization}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Contact (optional)
|
||||||
|
if (config.contact) {
|
||||||
|
const contactValue = config.contact.includes('@') && !config.contact.startsWith('mailto:')
|
||||||
|
? `mailto:${config.contact}`
|
||||||
|
: config.contact;
|
||||||
|
lines.push(`Contact: ${contactValue}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Verification URL (optional)
|
||||||
|
if (config.verification) {
|
||||||
|
lines.push(`Verification: ${config.verification}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Frequency (optional)
|
||||||
|
if (config.frequency) {
|
||||||
|
lines.push(`Frequency: ${config.frequency}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Previous canary (optional)
|
||||||
|
if (config.previousCanary) {
|
||||||
|
lines.push(`Previous-Canary: ${config.previousCanary}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Blockchain proof (optional)
|
||||||
|
if (config.blockchainProof) {
|
||||||
|
const { network, address, txHash, timestamp } = config.blockchainProof;
|
||||||
|
const proof = txHash
|
||||||
|
? `${network}:${address}:${txHash}`
|
||||||
|
: `${network}:${address}`;
|
||||||
|
lines.push(`Blockchain-Proof: ${proof}`);
|
||||||
|
if (timestamp) {
|
||||||
|
lines.push(`Blockchain-Timestamp: ${timestamp}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
lines.push('');
|
||||||
|
|
||||||
|
// Statement section
|
||||||
|
lines.push(`Statement: As of ${today}, ${config.organization || siteURL.hostname} has NOT received:`);
|
||||||
|
|
||||||
|
const statements = typeof config.statements === 'function'
|
||||||
|
? config.statements()
|
||||||
|
: config.statements || DEFAULT_STATEMENTS;
|
||||||
|
|
||||||
|
statements
|
||||||
|
.filter(s => !s.received)
|
||||||
|
.forEach(statement => {
|
||||||
|
lines.push(` - ${statement.description}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
lines.push('');
|
||||||
|
|
||||||
|
// Additional statement (optional)
|
||||||
|
if (config.additionalStatement) {
|
||||||
|
lines.push(config.additionalStatement.trim());
|
||||||
|
lines.push('');
|
||||||
|
}
|
||||||
|
|
||||||
|
// Key personnel statement (optional)
|
||||||
|
if (config.personnelStatement) {
|
||||||
|
lines.push('Key Personnel Statement: All key personnel with access to');
|
||||||
|
lines.push('infrastructure remain free and under no duress.');
|
||||||
|
lines.push('');
|
||||||
|
}
|
||||||
|
|
||||||
|
// Update reminder based on frequency
|
||||||
|
const frequencyText = config.frequency || 'monthly';
|
||||||
|
const expirationDays = Math.ceil((new Date(expires).getTime() - Date.now()) / (24 * 60 * 60 * 1000));
|
||||||
|
|
||||||
|
lines.push(`This canary will be updated ${frequencyText}. Absence of an update`);
|
||||||
|
lines.push(`within ${expirationDays} days should be considered significant.`);
|
||||||
|
lines.push('');
|
||||||
|
|
||||||
|
// Footer
|
||||||
|
lines.push('---');
|
||||||
|
lines.push('');
|
||||||
|
lines.push('This warrant canary follows the proposed canary.txt specification.');
|
||||||
|
lines.push('See: https://github.com/withastro/astro-discovery/blob/main/CANARY_SPEC.md');
|
||||||
|
|
||||||
|
return lines.join('\n').trim() + '\n';
|
||||||
|
}
|
||||||
75
src/generators/security.ts
Normal file
75
src/generators/security.ts
Normal file
@ -0,0 +1,75 @@
|
|||||||
|
import type { SecurityConfig } from '../types.js';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Generate security.txt content per RFC 9116
|
||||||
|
*
|
||||||
|
* @param config - Security.txt configuration
|
||||||
|
* @param siteURL - Site base URL
|
||||||
|
* @returns Generated security.txt content
|
||||||
|
*
|
||||||
|
* @see https://datatracker.ietf.org/doc/html/rfc9116
|
||||||
|
*/
|
||||||
|
export function generateSecurityTxt(
|
||||||
|
config: SecurityConfig,
|
||||||
|
siteURL: URL
|
||||||
|
): string {
|
||||||
|
const lines: string[] = [];
|
||||||
|
|
||||||
|
// Canonical URL (recommended)
|
||||||
|
const canonical = config.canonical || new URL('.well-known/security.txt', siteURL).href;
|
||||||
|
lines.push(`Canonical: ${canonical}`);
|
||||||
|
lines.push('');
|
||||||
|
|
||||||
|
// Contact (required) - can be email or URL
|
||||||
|
const contacts = Array.isArray(config.contact) ? config.contact : [config.contact];
|
||||||
|
contacts.forEach(contact => {
|
||||||
|
// Add mailto: prefix if it's an email
|
||||||
|
const contactValue = contact.includes('@') && !contact.startsWith('mailto:')
|
||||||
|
? `mailto:${contact}`
|
||||||
|
: contact;
|
||||||
|
lines.push(`Contact: ${contactValue}`);
|
||||||
|
});
|
||||||
|
lines.push('');
|
||||||
|
|
||||||
|
// Expires (required by RFC)
|
||||||
|
const expires = config.expires === 'auto'
|
||||||
|
? new Date(Date.now() + 365 * 24 * 60 * 60 * 1000).toISOString() // 1 year from now
|
||||||
|
: config.expires || new Date(Date.now() + 365 * 24 * 60 * 60 * 1000).toISOString();
|
||||||
|
lines.push(`Expires: ${expires}`);
|
||||||
|
lines.push('');
|
||||||
|
|
||||||
|
// Encryption (optional)
|
||||||
|
if (config.encryption) {
|
||||||
|
const encryptions = Array.isArray(config.encryption) ? config.encryption : [config.encryption];
|
||||||
|
encryptions.forEach(key => {
|
||||||
|
lines.push(`Encryption: ${key}`);
|
||||||
|
});
|
||||||
|
lines.push('');
|
||||||
|
}
|
||||||
|
|
||||||
|
// Acknowledgments (optional)
|
||||||
|
if (config.acknowledgments) {
|
||||||
|
lines.push(`Acknowledgments: ${config.acknowledgments}`);
|
||||||
|
lines.push('');
|
||||||
|
}
|
||||||
|
|
||||||
|
// Preferred-Languages (optional)
|
||||||
|
if (config.preferredLanguages && config.preferredLanguages.length > 0) {
|
||||||
|
lines.push(`Preferred-Languages: ${config.preferredLanguages.join(', ')}`);
|
||||||
|
lines.push('');
|
||||||
|
}
|
||||||
|
|
||||||
|
// Policy (optional)
|
||||||
|
if (config.policy) {
|
||||||
|
lines.push(`Policy: ${config.policy}`);
|
||||||
|
lines.push('');
|
||||||
|
}
|
||||||
|
|
||||||
|
// Hiring (optional)
|
||||||
|
if (config.hiring) {
|
||||||
|
lines.push(`Hiring: ${config.hiring}`);
|
||||||
|
lines.push('');
|
||||||
|
}
|
||||||
|
|
||||||
|
return lines.join('\n').trim() + '\n';
|
||||||
|
}
|
||||||
25
src/index.ts
25
src/index.ts
@ -103,6 +103,22 @@ export default function discovery(
|
|||||||
prerender: true
|
prerender: true
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (config.security && config.security.enabled !== false) {
|
||||||
|
injectRoute({
|
||||||
|
pattern: '/.well-known/security.txt',
|
||||||
|
entrypoint: '@astrojs/discovery/routes/security',
|
||||||
|
prerender: true
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
if (config.canary && config.canary.enabled !== false) {
|
||||||
|
injectRoute({
|
||||||
|
pattern: '/.well-known/canary.txt',
|
||||||
|
entrypoint: '@astrojs/discovery/routes/canary',
|
||||||
|
prerender: true
|
||||||
|
});
|
||||||
|
}
|
||||||
},
|
},
|
||||||
|
|
||||||
'astro:build:done': () => {
|
'astro:build:done': () => {
|
||||||
@ -118,6 +134,12 @@ export default function discovery(
|
|||||||
if (config.humans?.enabled !== false) {
|
if (config.humans?.enabled !== false) {
|
||||||
console.log(' ✅ /humans.txt');
|
console.log(' ✅ /humans.txt');
|
||||||
}
|
}
|
||||||
|
if (config.security && config.security.enabled !== false) {
|
||||||
|
console.log(' ✅ /.well-known/security.txt');
|
||||||
|
}
|
||||||
|
if (config.canary && config.canary.enabled !== false) {
|
||||||
|
console.log(' ✅ /.well-known/canary.txt');
|
||||||
|
}
|
||||||
|
|
||||||
console.log(' ✅ /sitemap-index.xml');
|
console.log(' ✅ /sitemap-index.xml');
|
||||||
console.log('');
|
console.log('');
|
||||||
@ -132,6 +154,9 @@ export type {
|
|||||||
RobotsConfig,
|
RobotsConfig,
|
||||||
LLMsConfig,
|
LLMsConfig,
|
||||||
HumansConfig,
|
HumansConfig,
|
||||||
|
SecurityConfig,
|
||||||
|
CanaryConfig,
|
||||||
|
CanaryStatement,
|
||||||
SitemapConfig,
|
SitemapConfig,
|
||||||
CachingConfig,
|
CachingConfig,
|
||||||
TemplateConfig,
|
TemplateConfig,
|
||||||
|
|||||||
30
src/routes/canary.ts
Normal file
30
src/routes/canary.ts
Normal file
@ -0,0 +1,30 @@
|
|||||||
|
import type { APIRoute } from 'astro';
|
||||||
|
import { generateCanaryTxt } from '../generators/canary.js';
|
||||||
|
// @ts-ignore - Virtual module
|
||||||
|
import config from 'virtual:@astrojs/discovery/config';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* API route for /.well-known/canary.txt (warrant canary)
|
||||||
|
*/
|
||||||
|
export const GET: APIRoute = ({ site }) => {
|
||||||
|
const canaryConfig = config.canary || {};
|
||||||
|
const siteURL = site || new URL('http://localhost:4321');
|
||||||
|
|
||||||
|
// Use custom template if provided
|
||||||
|
const content = config.templates?.canary
|
||||||
|
? config.templates.canary(canaryConfig, siteURL)
|
||||||
|
: generateCanaryTxt(canaryConfig, siteURL);
|
||||||
|
|
||||||
|
// Get cache duration (default: 1 hour - canaries should be checked frequently)
|
||||||
|
const cacheSeconds = config.caching?.canary ?? 3600;
|
||||||
|
|
||||||
|
return new Response(content, {
|
||||||
|
status: 200,
|
||||||
|
headers: {
|
||||||
|
'Content-Type': 'text/plain; charset=utf-8',
|
||||||
|
'Cache-Control': `public, max-age=${cacheSeconds}`,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
};
|
||||||
|
|
||||||
|
export const prerender = true;
|
||||||
30
src/routes/security.ts
Normal file
30
src/routes/security.ts
Normal file
@ -0,0 +1,30 @@
|
|||||||
|
import type { APIRoute } from 'astro';
|
||||||
|
import { generateSecurityTxt } from '../generators/security.js';
|
||||||
|
// @ts-ignore - Virtual module
|
||||||
|
import config from 'virtual:@astrojs/discovery/config';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* API route for /.well-known/security.txt (RFC 9116)
|
||||||
|
*/
|
||||||
|
export const GET: APIRoute = ({ site }) => {
|
||||||
|
const securityConfig = config.security || {};
|
||||||
|
const siteURL = site || new URL('http://localhost:4321');
|
||||||
|
|
||||||
|
// Use custom template if provided
|
||||||
|
const content = config.templates?.security
|
||||||
|
? config.templates.security(securityConfig, siteURL)
|
||||||
|
: generateSecurityTxt(securityConfig, siteURL);
|
||||||
|
|
||||||
|
// Get cache duration (default: 24 hours)
|
||||||
|
const cacheSeconds = config.caching?.security ?? 86400;
|
||||||
|
|
||||||
|
return new Response(content, {
|
||||||
|
status: 200,
|
||||||
|
headers: {
|
||||||
|
'Content-Type': 'text/plain; charset=utf-8',
|
||||||
|
'Cache-Control': `public, max-age=${cacheSeconds}`,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
};
|
||||||
|
|
||||||
|
export const prerender = true;
|
||||||
81
src/types.ts
81
src/types.ts
@ -8,6 +8,10 @@ export interface DiscoveryConfig {
|
|||||||
llms?: LLMsConfig;
|
llms?: LLMsConfig;
|
||||||
/** Configuration for humans.txt generation */
|
/** Configuration for humans.txt generation */
|
||||||
humans?: HumansConfig;
|
humans?: HumansConfig;
|
||||||
|
/** Configuration for security.txt generation (RFC 9116) */
|
||||||
|
security?: SecurityConfig;
|
||||||
|
/** Configuration for canary.txt generation (warrant canary) */
|
||||||
|
canary?: CanaryConfig;
|
||||||
/** Configuration passed to @astrojs/sitemap */
|
/** Configuration passed to @astrojs/sitemap */
|
||||||
sitemap?: SitemapConfig;
|
sitemap?: SitemapConfig;
|
||||||
/** HTTP cache control configuration */
|
/** HTTP cache control configuration */
|
||||||
@ -126,6 +130,10 @@ export interface CachingConfig {
|
|||||||
llms?: number;
|
llms?: number;
|
||||||
/** Cache duration for humans.txt (default: 86400) */
|
/** Cache duration for humans.txt (default: 86400) */
|
||||||
humans?: number;
|
humans?: number;
|
||||||
|
/** Cache duration for security.txt (default: 86400) */
|
||||||
|
security?: number;
|
||||||
|
/** Cache duration for canary.txt (default: 3600) */
|
||||||
|
canary?: number;
|
||||||
/** Cache duration for sitemap (default: 3600) */
|
/** Cache duration for sitemap (default: 3600) */
|
||||||
sitemap?: number;
|
sitemap?: number;
|
||||||
}
|
}
|
||||||
@ -140,6 +148,10 @@ export interface TemplateConfig {
|
|||||||
llms?: (config: LLMsConfig, siteURL: URL) => string | Promise<string>;
|
llms?: (config: LLMsConfig, siteURL: URL) => string | Promise<string>;
|
||||||
/** Custom humans.txt template */
|
/** Custom humans.txt template */
|
||||||
humans?: (config: HumansConfig, siteURL: URL) => string;
|
humans?: (config: HumansConfig, siteURL: URL) => string;
|
||||||
|
/** Custom security.txt template */
|
||||||
|
security?: (config: SecurityConfig, siteURL: URL) => string;
|
||||||
|
/** Custom canary.txt template */
|
||||||
|
canary?: (config: CanaryConfig, siteURL: URL) => string;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@ -220,6 +232,75 @@ export interface SiteInfo {
|
|||||||
software?: string[];
|
software?: string[];
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Configuration for security.txt generation (RFC 9116)
|
||||||
|
*/
|
||||||
|
export interface SecurityConfig {
|
||||||
|
/** Enable/disable security.txt generation (default: true) */
|
||||||
|
enabled?: boolean;
|
||||||
|
/** Contact email or URL for security issues (REQUIRED by RFC) */
|
||||||
|
contact: string | string[];
|
||||||
|
/** Expiration date (ISO 8601 or 'auto' for 1 year) */
|
||||||
|
expires?: string | 'auto';
|
||||||
|
/** URL to encryption key (PGP public key) */
|
||||||
|
encryption?: string | string[];
|
||||||
|
/** URL to security acknowledgments/hall of fame */
|
||||||
|
acknowledgments?: string;
|
||||||
|
/** Preferred languages for security reports */
|
||||||
|
preferredLanguages?: string[];
|
||||||
|
/** URL for canonical security.txt */
|
||||||
|
canonical?: string;
|
||||||
|
/** Security policy URL */
|
||||||
|
policy?: string;
|
||||||
|
/** Hiring/jobs URL for security team */
|
||||||
|
hiring?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Configuration for canary.txt generation (warrant canary)
|
||||||
|
*/
|
||||||
|
export interface CanaryConfig {
|
||||||
|
/** Enable/disable canary.txt generation (default: true) */
|
||||||
|
enabled?: boolean;
|
||||||
|
/** Organization name */
|
||||||
|
organization?: string;
|
||||||
|
/** Contact email for canary inquiries */
|
||||||
|
contact?: string;
|
||||||
|
/** Update frequency (daily, weekly, monthly, quarterly, yearly) */
|
||||||
|
frequency?: 'daily' | 'weekly' | 'monthly' | 'quarterly' | 'yearly';
|
||||||
|
/** Expiration date (ISO 8601 or 'auto' based on frequency) */
|
||||||
|
expires?: string | 'auto';
|
||||||
|
/** Statements about what has NOT been received */
|
||||||
|
statements?: CanaryStatement[] | (() => CanaryStatement[]);
|
||||||
|
/** Additional free-form statement text */
|
||||||
|
additionalStatement?: string;
|
||||||
|
/** URL to PGP signature or verification */
|
||||||
|
verification?: string;
|
||||||
|
/** URL to previous canary (for continuity) */
|
||||||
|
previousCanary?: string;
|
||||||
|
/** Blockchain proof information */
|
||||||
|
blockchainProof?: {
|
||||||
|
network: string;
|
||||||
|
address: string;
|
||||||
|
txHash?: string;
|
||||||
|
timestamp?: string;
|
||||||
|
};
|
||||||
|
/** Include key personnel statement */
|
||||||
|
personnelStatement?: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Individual canary statement
|
||||||
|
*/
|
||||||
|
export interface CanaryStatement {
|
||||||
|
/** Type of request/order */
|
||||||
|
type: 'nsl' | 'fisa' | 'gag' | 'surveillance' | 'backdoor' | 'encryption' | 'other';
|
||||||
|
/** Human-readable description */
|
||||||
|
description: string;
|
||||||
|
/** Whether this type of request has been received */
|
||||||
|
received: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Sitemap item interface (from @astrojs/sitemap)
|
* Sitemap item interface (from @astrojs/sitemap)
|
||||||
*/
|
*/
|
||||||
|
|||||||
@ -3,7 +3,7 @@ import type { DiscoveryConfig } from '../types.js';
|
|||||||
/**
|
/**
|
||||||
* Default configuration values
|
* Default configuration values
|
||||||
*/
|
*/
|
||||||
const DEFAULT_CONFIG: Required<Omit<DiscoveryConfig, 'templates'>> & { templates?: DiscoveryConfig['templates'] } = {
|
const DEFAULT_CONFIG: Required<Omit<DiscoveryConfig, 'templates' | 'security' | 'canary'>> & { templates?: DiscoveryConfig['templates']; security?: DiscoveryConfig['security']; canary?: DiscoveryConfig['canary'] } = {
|
||||||
robots: {
|
robots: {
|
||||||
enabled: true,
|
enabled: true,
|
||||||
crawlDelay: 1,
|
crawlDelay: 1,
|
||||||
@ -23,6 +23,8 @@ const DEFAULT_CONFIG: Required<Omit<DiscoveryConfig, 'templates'>> & { templates
|
|||||||
robots: 3600, // 1 hour
|
robots: 3600, // 1 hour
|
||||||
llms: 3600, // 1 hour
|
llms: 3600, // 1 hour
|
||||||
humans: 86400, // 24 hours
|
humans: 86400, // 24 hours
|
||||||
|
security: 86400, // 24 hours
|
||||||
|
canary: 3600, // 1 hour
|
||||||
sitemap: 3600, // 1 hour
|
sitemap: 3600, // 1 hour
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
@ -51,6 +53,8 @@ export function validateConfig(userConfig: DiscoveryConfig = {}): DiscoveryConfi
|
|||||||
...DEFAULT_CONFIG.humans,
|
...DEFAULT_CONFIG.humans,
|
||||||
...userConfig.humans,
|
...userConfig.humans,
|
||||||
},
|
},
|
||||||
|
security: userConfig.security,
|
||||||
|
canary: userConfig.canary,
|
||||||
sitemap: {
|
sitemap: {
|
||||||
...DEFAULT_CONFIG.sitemap,
|
...DEFAULT_CONFIG.sitemap,
|
||||||
...userConfig.sitemap,
|
...userConfig.sitemap,
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user