Ryan Malloy
d25dde4627
This commit introduces a comprehensive Astro integration that automatically generates discovery files for websites: Features: - robots.txt with LLM bot support (Anthropic-AI, GPTBot, etc.) - llms.txt for AI assistant context and instructions - humans.txt for team credits and site information - Automatic sitemap integration via @astrojs/sitemap Technical Details: - TypeScript implementation with full type safety - Configurable HTTP caching headers - Custom template support for all generated files - Sensible defaults with extensive customization options - Date-based versioning (2025.11.03) Testing: - 34 unit tests covering all generators - Test coverage for robots.txt, llms.txt, and humans.txt - Integration with Vitest Documentation: - Comprehensive README with examples - API reference documentation - Contributing guidelines - Example configurations (minimal and full)
19 lines
472 B
TypeScript
19 lines
472 B
TypeScript
import { defineConfig } from 'astro/config';
|
|
import discovery from '@astrojs/discovery';
|
|
|
|
// Minimal configuration - just provide your site URL
|
|
// Everything else uses sensible defaults
|
|
export default defineConfig({
|
|
site: 'https://example.com',
|
|
|
|
integrations: [
|
|
discovery(),
|
|
],
|
|
});
|
|
|
|
// This will generate:
|
|
// - /robots.txt with LLM bot support
|
|
// - /llms.txt with basic site info
|
|
// - /humans.txt with basic structure
|
|
// - /sitemap-index.xml with all your pages
|