ivatar/.cursorrules

# ivatar/libravatar Project Rules

## Project Overview
ivatar is a Django-based federated avatar service that serves as an alternative to Gravatar. It provides avatar images for email addresses and OpenID URLs, with support for the Libravatar federation protocol.

## Core Functionality
- Avatar service for email addresses and OpenID URLs
- Federated compatibility with Libravatar protocol
- Multiple authentication methods (OpenID, OpenID Connect/Fedora, Django auth)
- Image upload, cropping, and management
- External avatar import (Gravatar, other Libravatar instances)
- Bluesky handle integration
- Multiple theme support (default, clime, green, red)
- Internationalization (15+ languages)

## Technical Stack
- **Framework**: Django 4.2+ with Python 3.x
- **Database**: SQLite (development), MySQL/MariaDB, PostgreSQL (production)
- **Image Processing**: PIL/Pillow for image manipulation
- **Authentication**: django-openid-auth, social-auth-app-django
- **Caching**: Memcached and filesystem caching
- **Email**: Mailgun integration via django-anymail
- **Testing**: pytest with custom markers

## Key Models
- `Photo`: Stores uploaded avatar images with format detection and access counting
- `ConfirmedEmail`: Verified email addresses with assigned photos and Bluesky handles
- `ConfirmedOpenId`: Verified OpenID URLs with assigned photos and Bluesky handles
- `UserPreference`: User theme preferences
- `UnconfirmedEmail`: Email verification workflow
- `UnconfirmedOpenId`: OpenID verification workflow

## Security Features
- File upload validation and sanitization
- EXIF data removal (ENABLE_EXIF_SANITIZATION)
- Malicious content scanning (ENABLE_MALICIOUS_CONTENT_SCAN)
- Comprehensive security logging
- File size limits and format validation
- Trusted URL validation for external avatar sources

## Development Workflow Rules

### External Resources & Libraries
- **Web search is always allowed** - use web search to find solutions, check documentation, verify best practices
- **Use latest library versions** - always prefer the latest stable versions of external libraries
- **Security first** - outdated libraries are security risks, always update to latest versions
- **Dependency management** - when adding new dependencies, ensure they're actively maintained and secure

### Testing
- **MANDATORY: Run pre-commit hooks and tests before any changes** - this is an obligation
- Use `./run_tests_local.sh` for local development (skips Bluesky tests requiring API credentials)
- Run `python3 manage.py test -v3` for full test suite including Bluesky tests
- **MANDATORY: When adding new code, always write tests to increase code coverage** - never decrease coverage
- Use pytest markers appropriately:
  - `@pytest.mark.bluesky`: Tests requiring Bluesky API credentials
  - `@pytest.mark.slow`: Long-running tests
  - `@pytest.mark.integration`: Integration tests
  - `@pytest.mark.unit`: Unit tests

### Code Quality
- Always check for linter errors after making changes using `read_lints`
- Follow existing code style and patterns
- Maintain comprehensive logging (use `logger = logging.getLogger("ivatar")`)
- Consider security implications of any changes
- Follow Django best practices and conventions
- **Reduce script creation** - avoid creating unnecessary scripts, prefer existing tools and commands
- **Use latest libraries** - always use the latest versions of external libraries to ensure security and bug fixes

### Database Operations
- Use migrations for schema changes: `./manage.py migrate`
- Support multiple database backends (SQLite, MySQL, PostgreSQL)
- Use proper indexing for performance (see existing model indexes)

### Image Processing
- Support multiple formats: JPEG, PNG, GIF, WEBP
- Maximum image size: 512x512 pixels (AVATAR_MAX_SIZE)
- Maximum file size: 10MB (MAX_PHOTO_SIZE)
- JPEG quality: 85 (JPEG_QUALITY)
- Always validate image format and dimensions

## Configuration Management
- Main settings in `ivatar/settings.py` and `config.py`
- Local overrides in `config_local.py` (not in version control)
- Environment variables for sensitive data (database credentials, API keys)
- Support for multiple deployment environments (development, staging, production)

## Authentication & Authorization
- Multiple backends: Django auth, OpenID, Fedora OIDC
- Social auth pipeline with custom steps for email confirmation
- User account creation and management
- Email verification workflow

## Caching Strategy
- Memcached for general caching
- Filesystem cache for generated images
- 5-minute cache for resized images (CACHE_IMAGES_MAX_AGE)
- Cache invalidation on photo updates

## Internationalization
- Support for 15+ languages
- Use Django's translation framework
- Template strings should be translatable
- Locale-specific formatting

## File Structure Guidelines
- Main Django app: `ivatar/`
- Account management: `ivatar/ivataraccount/`
- Tools: `ivatar/tools/`
- Static files: `ivatar/static/` and `static/`
- Templates: `templates/` and app-specific template directories
- Tests: Co-located with modules or in dedicated test files

## Security Considerations
- Always validate file uploads
- Sanitize EXIF data from images
- Use secure password hashing (Argon2 preferred, PBKDF2 fallback)
- Implement proper CSRF protection
- Use secure cookies in production
- Log security events to dedicated security log

## Performance Considerations
- Use database indexes for frequently queried fields
- Implement proper caching strategies
- Optimize image processing operations
- Monitor access counts for analytics
- Use efficient database queries

## Production Deployment & Infrastructure

### Hosting & Sponsorship
- **Hosted by Fedora Project** - Free infrastructure provided due to heavy usage by Fedora community
- **Scale**: Handles millions of requests daily for 30k+ users with 33k+ avatar images
- **Performance**: High-performance system optimized for dynamic content (CDN difficult due to dynamic sizing)

### Production Architecture
- **Redis**: Session storage (potential future caching expansion)
- **Monitoring Stack**:
  - Prometheus + Alertmanager for metrics and alerting
  - Loki for log aggregation
  - Alloy for observability
  - Grafana for visualization
  - Custom exporters for application metrics
- **Apache HTTPD**:
  - SSL termination
  - Load balancer for Gunicorn containers
  - Caching (memory/socache and disk cache - optimization ongoing)
- **PostgreSQL**: Main production database
- **Gunicorn**: 2 containers running Django application
- **Containerization**: **Podman** (not Docker) - always prefer podman when possible

### Development Environment
- **Dev Instance**: dev.libravatar.org (auto-deployed from 'devel' branch via Puppet)
- **Limitation**: Aging CentOS 7 host with older Python 3.x and Django versions
- **Compatibility**: Must maintain backward compatibility with older versions

### CI/CD & Version Control
- **GitLab**: Self-hosted OSS/Community Edition on git.linux-kernel.at
- **CI**: GitLab CI extensively used
- **CD**: GitLab CD on roadmap (part of libravatar-ansible project)
- **Deployment**: Separate libravatar-ansible project handles production deployments
- **Container Management**: Ansible playbooks rebuild custom images and restart containers as needed

### Deployment Considerations
- Production requires proper database setup (PostgreSQL, not SQLite)
- Static file collection required: `./manage.py collectstatic`
- Environment-specific configuration via environment variables
- Custom container images with automated rebuilds
- High availability and performance optimization critical

## Common Commands
```bash
# Development server
./manage.py runserver 0:8080

# Run local tests (recommended for development)
./run_tests_local.sh

# Run all tests
python3 manage.py test -v2

# Database migrations
./manage.py migrate

# Collect static files
./manage.py collectstatic -l --no-input

# Create superuser
./manage.py createsuperuser
```

## Code Style Guidelines
- Use descriptive variable and function names
- Add comprehensive docstrings for classes and methods
- **MANDATORY: Include type hints for ALL new code** - this is a strict requirement
- Follow PEP 8 and Django coding standards
- Use meaningful commit messages
- Add comments for complex business logic

## Error Handling
- Use proper exception handling with specific exception types
- Log errors with appropriate levels (DEBUG, INFO, WARNING, ERROR)
- Provide user-friendly error messages
- Implement graceful fallbacks where possible

## API Compatibility
- Maintain backward compatibility with existing avatar URLs
- Support Libravatar federation protocol
- Ensure Gravatar compatibility for imports
- Preserve existing URL patterns and parameters

## Monitoring & Logging
- Use structured logging with appropriate levels
- Log security events to dedicated security log
- Monitor performance metrics (access counts, response times)
- Implement health checks for external dependencies
- **Robust logging setup**: Automatically tests directory writeability and falls back gracefully
- **Fallback hierarchy**: logs/ → /tmp/libravatar-logs → user-specific temp directory
- **Permission handling**: Handles cases where logs directory exists but isn't writable

## GitLab CI/CD Monitoring
- **MANDATORY: Check GitLab pipeline status regularly** during development
- Monitor pipeline status for the current working branch (typically `devel`)
- Use `glab ci list --repo git.linux-kernel.at/oliver/ivatar --per-page 5` to check recent pipelines
- Verify all tests pass before considering work complete
- Check pipeline logs with `glab ci trace <pipeline-id> --repo git.linux-kernel.at/oliver/ivatar` if needed
- Address any CI failures immediately before proceeding with new changes
- Pipeline URL: https://git.linux-kernel.at/oliver/ivatar/-/pipelines

## Deployment Verification
- **Automatic verification**: GitLab CI automatically verifies dev.libravatar.org deployments on `devel` branch
- **Manual verification**: Production deployments on `master` branch can be verified manually via CI
- **Version endpoint**: `/deployment/version/` provides commit hash, branch, and deployment status
- **Security**: Version endpoint uses cached git file reading (no subprocess calls) to prevent DDoS attacks
- **Performance**: Version information is cached in memory to avoid repeated file system access
- **SELinux compatibility**: No subprocess calls that might be blocked by SELinux policies
- **Manual testing**: Use `./scripts/test_deployment.sh` to test deployments locally
- **Deployment timing**: Dev deployments via Puppet may take up to 30 minutes to complete
- **Verification includes**: Version matching, avatar endpoint, stats endpoint functionality

Remember: This is a production avatar service handling user data and images. Security, performance, and reliability are paramount. Always consider the impact of changes on existing users and federated services.