# Example Generated Files This document shows what the generated discovery files look like. ## Example robots.txt ``` # robots.txt # Generated by @astrojs/discovery for example.com User-agent: * Allow: / # Sitemaps Sitemap: https://example.com/sitemap-index.xml # LLM-specific resources # AI assistants can find additional context at /llms.txt # See: https://github.com/anthropics/llm-txt User-agent: Anthropic-AI User-agent: Claude-Web User-agent: GPTBot User-agent: ChatGPT-User User-agent: cohere-ai User-agent: Google-Extended User-agent: PerplexityBot User-agent: Applebot-Extended Allow: /llms.txt Allow: /llms-full.txt # Crawl delay (be nice to our server) Crawl-delay: 1 ``` ## Example llms.txt ``` # example.com > An awesome web development platform built with Astro --- ## Site Information - **URL**: https://example.com/ - **Description**: An awesome web development platform built with Astro ## Key Features - Lightning-fast static site generation - Built-in SEO optimization - AI-powered content suggestions - Modern component architecture ## Important Pages - **[Documentation](https://example.com/docs)** Complete API documentation and guides - **[Blog](https://example.com/blog)** Latest articles and tutorials - **[Examples](https://example.com/examples)** Real-world code examples ## Instructions for AI Assistants When helping users with our site: 1. Check documentation first at /docs 2. Use provided API endpoints for dynamic queries 3. Follow brand guidelines for tone and style 4. Be helpful, accurate, and concise 5. Provide code examples when relevant ## API Endpoints - `POST /api/chat` Chat endpoint for real-time conversations Full URL: https://example.com/api/chat - `GET /api/search` Search API for site content Full URL: https://example.com/api/search - `POST /api/generate` AI content generation endpoint Full URL: https://example.com/api/generate ## Technical Stack - **Frontend**: Astro, TypeScript, React, Tailwind CSS - **Backend**: Node.js, FastAPI, PostgreSQL - **AI/ML**: Claude, GPT-4, LangChain - **Other**: Docker, Redis, Nginx ## Brand Voice & Guidelines - Professional yet friendly and approachable - Technical but accessible to beginners - Focus on practical, real-world examples - Encourage experimentation and learning - Community-focused and inclusive --- **Last Updated**: 2025-11-03 *This file was generated by [@astrojs/discovery](https://github.com/withastro/astro-discovery)* ``` ## Example humans.txt ``` /* TEAM */ Name: Jane Doe Role: Creator & Lead Developer Contact: jane@example.com From: San Francisco, CA Twitter: @janedoe GitHub: janedoe Name: John Smith Role: Designer & UX Architect Contact: john@example.com From: New York, NY Twitter: @johnsmith GitHub: johnsmith /* THANKS */ The Astro team for creating amazing tools Open source community for endless inspiration Coffee ☕ for making this all possible Stack Overflow for solving impossible bugs /* SITE */ Last update: 2025-11-03 Language: English Doctype: HTML5 IDE: VS Code Tech Stack: Astro, TypeScript, React, Tailwind CSS, Node.js Standards: HTML5, CSS3, ES2022, WCAG 2.1 Components: Astro Components, React Components, Custom Web Components Software: VS Code, Git, Docker, Figma /* THE STORY */ This project started in early 2025 when we realized that the web needed better discovery mechanisms. Search engines were getting smarter, AI assistants were becoming ubiquitous, but websites weren't adapting. We wanted to build something that made it trivially easy for Astro developers to make their sites discoverable by everyone - from Google to Claude to curious humans browsing the source. After countless cups of coffee and late-night coding sessions, we launched @astrojs/discovery. It's been an incredible journey! /* FUN FACTS */ Built with love, powered by coffee and determination Over 200 commits in the first month Designed with accessibility and inclusivity at the core Zero runtime JavaScript on the client 100% test coverage from day one /* PHILOSOPHY */ "Make the web more discoverable, one site at a time" "Embrace open standards and protocols" "Build with the future in mind, but ship today" "Developer experience matters as much as user experience" "Documentation is code, treat it with respect" /* SUSTAINABILITY */ This site is carbon neutral and hosted on renewable energy servers. We measure our environmental impact and offset our carbon emissions. Building a better web means building a sustainable web. ``` ## Example sitemap-index.xml ```xml https://example.com/sitemap-0.xml ``` --- These files are automatically generated based on your configuration in `astro.config.mjs`. Customize them by adjusting the configuration options!