astro-discovery/CONTRIBUTING.md
Ryan Malloy d25dde4627 feat: initial implementation of @astrojs/discovery integration
This commit introduces a comprehensive Astro integration that automatically
generates discovery files for websites:

Features:
- robots.txt with LLM bot support (Anthropic-AI, GPTBot, etc.)
- llms.txt for AI assistant context and instructions
- humans.txt for team credits and site information
- Automatic sitemap integration via @astrojs/sitemap

Technical Details:
- TypeScript implementation with full type safety
- Configurable HTTP caching headers
- Custom template support for all generated files
- Sensible defaults with extensive customization options
- Date-based versioning (2025.11.03)

Testing:
- 34 unit tests covering all generators
- Test coverage for robots.txt, llms.txt, and humans.txt
- Integration with Vitest

Documentation:
- Comprehensive README with examples
- API reference documentation
- Contributing guidelines
- Example configurations (minimal and full)
2025-11-03 07:36:39 -07:00

3.9 KiB

Contributing to @astrojs/discovery

Thank you for your interest in contributing to @astrojs/discovery! This guide will help you get started.

Development Setup

  1. Clone the repository

    git clone https://github.com/withastro/astro-discovery.git
    cd astro-discovery
    
  2. Install dependencies

    npm install
    
  3. Build the project

    npm run build
    
  4. Run tests

    npm test
    

Project Structure

@astrojs/discovery/
├── src/
│   ├── index.ts              # Main integration entry point
│   ├── types.ts              # TypeScript type definitions
│   ├── config-store.ts       # Global config management
│   ├── generators/
│   │   ├── robots.ts        # robots.txt generator
│   │   ├── llms.ts          # llms.txt generator
│   │   └── humans.ts        # humans.txt generator
│   ├── routes/
│   │   ├── robots.ts        # /robots.txt API route
│   │   ├── llms.ts          # /llms.txt API route
│   │   └── humans.ts        # /humans.txt API route
│   └── validators/
│       └── config.ts         # Configuration validation
├── dist/                     # Built output (generated)
├── example/                  # Example configurations
└── tests/                    # Test files (to be added)

Making Changes

Adding New Features

  1. Create a new branch for your feature

    git checkout -b feature/your-feature-name
    
  2. Make your changes following the existing code style

  3. Add tests for your changes

  4. Update documentation in README.md

  5. Build and test

    npm run build
    npm test
    

Code Style

  • Use TypeScript for all code
  • Follow existing naming conventions
  • Add JSDoc comments for public APIs
  • Keep functions focused and small
  • Use meaningful variable names

Commit Messages

Follow conventional commit format:

  • feat: New features
  • fix: Bug fixes
  • docs: Documentation changes
  • test: Test additions/changes
  • refactor: Code refactoring
  • chore: Maintenance tasks

Example:

feat: add support for custom LLM bot agents

- Added ability to specify custom LLM bot user agents
- Updated documentation with examples
- Added tests for custom agent configuration

Testing

Running Tests

# Run all tests
npm test

# Run tests in watch mode
npm run test:watch

# Run tests with coverage
npm run test:coverage

Writing Tests

Place tests in the tests/ directory. Follow these patterns:

import { describe, it, expect } from 'vitest';
import { generateRobotsTxt } from '../src/generators/robots';

describe('generateRobotsTxt', () => {
  it('generates basic robots.txt', () => {
    const result = generateRobotsTxt({}, new URL('https://example.com'));
    expect(result).toContain('User-agent: *');
    expect(result).toContain('Sitemap: https://example.com/sitemap-index.xml');
  });
});

Pull Request Process

  1. Update documentation: Ensure README.md reflects any changes

  2. Add tests: All new features should have tests

  3. Update CHANGELOG: Add your changes to CHANGELOG.md

  4. Create a pull request:

    • Use a clear, descriptive title
    • Reference any related issues
    • Describe your changes in detail
    • Include screenshots for UI changes
  5. Address review feedback: Be responsive to code review comments

Release Process

(For maintainers)

  1. Update version in package.json using date-based versioning (YYYY.MM.DD)
  2. Update CHANGELOG.md
  3. Create a git tag
  4. Push to npm
npm version 2025.11.03
npm publish

Questions?

  • Open an issue for bugs or feature requests
  • Start a discussion for questions or ideas
  • Join our Discord community

License

By contributing, you agree that your contributions will be licensed under the MIT License.