astro-discovery/docs/README.md

@astrojs/discovery Documentation

Comprehensive documentation for the @astrojs/discovery integration - a complete solution for generating discovery files (robots.txt, llms.txt, humans.txt, security.txt, canary.txt, webfinger, and sitemap) in your Astro projects.

Documentation Structure

This documentation follows the Diátaxis framework, organizing content into four distinct categories:

📚 Tutorials (Learning-oriented)

Step-by-step lessons to get started:

• Basic Setup
• Configure robots.txt
• Setup llms.txt
• Create humans.txt
• Security & Canary Files
• WebFinger Discovery

🛠️ How-to Guides (Task-oriented)

Practical guides for specific tasks:

• Block Specific Bots
• Customize LLM Instructions
• Add Team Members
• Filter Sitemap Pages
• Set Cache Headers
• Environment-specific Config
• Use with Content Collections
• Custom Templates
• ActivityPub Integration

📖 Reference (Information-oriented)

Technical specifications and API documentation:

• Configuration Options
• API Reference
• File-specific Options (robots, llms, humans, security, canary, webfinger, sitemap)
• Cache Options
• TypeScript Types

💡 Explanation (Understanding-oriented)

Background and conceptual information:

• Why Use Discovery Files?
• Understanding each file type
• SEO & Discoverability
• AI Assistant Integration
• Architecture & Design

Project Structure

docs/
├── public/              # Static assets
├── src/
│   ├── assets/         # Images and logos
│   │   ├── logo.svg
│   │   └── houston.webp
│   ├── content/
│   │   └── docs/       # Documentation content (Markdown/MDX)
│   │       ├── getting-started/
│   │       ├── tutorials/
│   │       ├── how-to/
│   │       ├── reference/
│   │       ├── explanation/
│   │       ├── examples/
│   │       ├── community/
│   │       └── index.mdx
│   └── styles/
│       └── custom.css  # Custom styling
├── astro.config.mjs    # Astro + Starlight config
├── package.json
└── tsconfig.json

Commands

All commands are run from the docs directory:

Command	Action
npm install	Install dependencies
npm run dev	Start dev server at localhost:4321
npm run build	Build production site to ./dist/
npm run preview	Preview build locally
npm run check	Run Astro type checking

Development

1. Install dependencies:

   npm install
   

2. Start development server:

   npm run dev
   

3. Visit http://localhost:4321 to see your changes live

Adding Documentation

Create a new page

Add a new .md or .mdx file in the appropriate category:

# For a tutorial
touch src/content/docs/tutorials/my-new-tutorial.md

# For a how-to guide
touch src/content/docs/how-to/my-new-guide.md

Add frontmatter

Every documentation page should have frontmatter:

---
title: Page Title
description: Brief description for SEO and page previews
---

Your content here...

Update sidebar

Add your page to the sidebar in astro.config.mjs:

sidebar: [
  {
    label: 'Tutorials',
    items: [
      { label: 'My New Tutorial', slug: 'tutorials/my-new-tutorial' },
    ],
  },
]

Features

• Diátaxis Framework: Well-organized documentation structure
• Search: Full-text search powered by Pagefind
• Dark Mode: Automatic dark/light theme switching
• Mobile Responsive: Optimized for all screen sizes
• Edit on GitHub: Direct links to edit pages
• Type-safe: TypeScript throughout
• Fast: Static site generation with Astro
• Accessible: WCAG 2.1 AA compliant

Deployment

The documentation can be deployed to any static hosting platform:

• Vercel: vercel deploy
• Netlify: netlify deploy
• GitHub Pages: Configure in repository settings
• Cloudflare Pages: Connect to GitHub repository

Contributing

To contribute to the documentation:

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Ensure the build succeeds (npm run build)
5. Submit a pull request

Resources

• Starlight Documentation
• Astro Documentation
• Diátaxis Framework
• @astrojs/discovery Repository

License

MIT