Ryan Malloy
d25dde4627
This commit introduces a comprehensive Astro integration that automatically generates discovery files for websites: Features: - robots.txt with LLM bot support (Anthropic-AI, GPTBot, etc.) - llms.txt for AI assistant context and instructions - humans.txt for team credits and site information - Automatic sitemap integration via @astrojs/sitemap Technical Details: - TypeScript implementation with full type safety - Configurable HTTP caching headers - Custom template support for all generated files - Sensible defaults with extensive customization options - Date-based versioning (2025.11.03) Testing: - 34 unit tests covering all generators - Test coverage for robots.txt, llms.txt, and humans.txt - Integration with Vitest Documentation: - Comprehensive README with examples - API reference documentation - Contributing guidelines - Example configurations (minimal and full)
170 lines
3.9 KiB
Markdown
170 lines
3.9 KiB
Markdown
# Contributing to @astrojs/discovery
|
|
|
|
Thank you for your interest in contributing to @astrojs/discovery! This guide will help you get started.
|
|
|
|
## Development Setup
|
|
|
|
1. **Clone the repository**
|
|
```bash
|
|
git clone https://github.com/withastro/astro-discovery.git
|
|
cd astro-discovery
|
|
```
|
|
|
|
2. **Install dependencies**
|
|
```bash
|
|
npm install
|
|
```
|
|
|
|
3. **Build the project**
|
|
```bash
|
|
npm run build
|
|
```
|
|
|
|
4. **Run tests**
|
|
```bash
|
|
npm test
|
|
```
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
@astrojs/discovery/
|
|
├── src/
|
|
│ ├── index.ts # Main integration entry point
|
|
│ ├── types.ts # TypeScript type definitions
|
|
│ ├── config-store.ts # Global config management
|
|
│ ├── generators/
|
|
│ │ ├── robots.ts # robots.txt generator
|
|
│ │ ├── llms.ts # llms.txt generator
|
|
│ │ └── humans.ts # humans.txt generator
|
|
│ ├── routes/
|
|
│ │ ├── robots.ts # /robots.txt API route
|
|
│ │ ├── llms.ts # /llms.txt API route
|
|
│ │ └── humans.ts # /humans.txt API route
|
|
│ └── validators/
|
|
│ └── config.ts # Configuration validation
|
|
├── dist/ # Built output (generated)
|
|
├── example/ # Example configurations
|
|
└── tests/ # Test files (to be added)
|
|
```
|
|
|
|
## Making Changes
|
|
|
|
### Adding New Features
|
|
|
|
1. Create a new branch for your feature
|
|
```bash
|
|
git checkout -b feature/your-feature-name
|
|
```
|
|
|
|
2. Make your changes following the existing code style
|
|
|
|
3. Add tests for your changes
|
|
|
|
4. Update documentation in README.md
|
|
|
|
5. Build and test
|
|
```bash
|
|
npm run build
|
|
npm test
|
|
```
|
|
|
|
### Code Style
|
|
|
|
- Use TypeScript for all code
|
|
- Follow existing naming conventions
|
|
- Add JSDoc comments for public APIs
|
|
- Keep functions focused and small
|
|
- Use meaningful variable names
|
|
|
|
### Commit Messages
|
|
|
|
Follow conventional commit format:
|
|
- `feat:` New features
|
|
- `fix:` Bug fixes
|
|
- `docs:` Documentation changes
|
|
- `test:` Test additions/changes
|
|
- `refactor:` Code refactoring
|
|
- `chore:` Maintenance tasks
|
|
|
|
Example:
|
|
```
|
|
feat: add support for custom LLM bot agents
|
|
|
|
- Added ability to specify custom LLM bot user agents
|
|
- Updated documentation with examples
|
|
- Added tests for custom agent configuration
|
|
```
|
|
|
|
## Testing
|
|
|
|
### Running Tests
|
|
|
|
```bash
|
|
# Run all tests
|
|
npm test
|
|
|
|
# Run tests in watch mode
|
|
npm run test:watch
|
|
|
|
# Run tests with coverage
|
|
npm run test:coverage
|
|
```
|
|
|
|
### Writing Tests
|
|
|
|
Place tests in the `tests/` directory. Follow these patterns:
|
|
|
|
```typescript
|
|
import { describe, it, expect } from 'vitest';
|
|
import { generateRobotsTxt } from '../src/generators/robots';
|
|
|
|
describe('generateRobotsTxt', () => {
|
|
it('generates basic robots.txt', () => {
|
|
const result = generateRobotsTxt({}, new URL('https://example.com'));
|
|
expect(result).toContain('User-agent: *');
|
|
expect(result).toContain('Sitemap: https://example.com/sitemap-index.xml');
|
|
});
|
|
});
|
|
```
|
|
|
|
## Pull Request Process
|
|
|
|
1. **Update documentation**: Ensure README.md reflects any changes
|
|
|
|
2. **Add tests**: All new features should have tests
|
|
|
|
3. **Update CHANGELOG**: Add your changes to CHANGELOG.md
|
|
|
|
4. **Create a pull request**:
|
|
- Use a clear, descriptive title
|
|
- Reference any related issues
|
|
- Describe your changes in detail
|
|
- Include screenshots for UI changes
|
|
|
|
5. **Address review feedback**: Be responsive to code review comments
|
|
|
|
## Release Process
|
|
|
|
(For maintainers)
|
|
|
|
1. Update version in package.json using date-based versioning (YYYY.MM.DD)
|
|
2. Update CHANGELOG.md
|
|
3. Create a git tag
|
|
4. Push to npm
|
|
|
|
```bash
|
|
npm version 2025.11.03
|
|
npm publish
|
|
```
|
|
|
|
## Questions?
|
|
|
|
- Open an issue for bugs or feature requests
|
|
- Start a discussion for questions or ideas
|
|
- Join our Discord community
|
|
|
|
## License
|
|
|
|
By contributing, you agree that your contributions will be licensed under the MIT License.
|