Mondo Diaz
2dea63f99f
Major enhancements: - Feature flag system for cloud vs air-gapped deployment modes - Automatic storage backend selection based on deployment mode - Comprehensive seed data generation utilities - Support for generating CSV, JSON, binary, and PCAP test files - Quick seed script for easy data generation - Angular 19 frontend complete setup documentation - Material Design UI component examples and configuration Fixes: - Resolve SQLAlchemy metadata column name conflict - Rename metadata to custom_metadata throughout codebase - Fix API health check issues Documentation: - FEATURES.md - Complete feature overview - FRONTEND_SETUP.md - Angular 19 setup guide with examples - SUMMARY.md - Implementation summary 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
232 lines
5.8 KiB
Markdown
232 lines
5.8 KiB
Markdown
# Features Overview
|
|
|
|
## Core Features
|
|
|
|
### Storage & Backend
|
|
- **Multi-format Support**: CSV, JSON, binary files, and PCAP (packet capture) files
|
|
- **Dual Storage Backend**:
|
|
- **AWS S3** for cloud deployments
|
|
- **MinIO** for air-gapped/self-hosted deployments
|
|
- **Automatic Backend Selection**: Based on deployment mode feature flag
|
|
- **Storage Abstraction**: Seamlessly switch between S3 and MinIO via configuration
|
|
|
|
### Database & Metadata
|
|
- **PostgreSQL Database**: Stores all artifact metadata
|
|
- **Rich Metadata Support**:
|
|
- Test information (name, suite, configuration, result)
|
|
- Custom metadata (JSON format)
|
|
- Tags for categorization
|
|
- File versioning support
|
|
- Timestamps and audit trail
|
|
|
|
### API Features
|
|
- **RESTful API**: Built with FastAPI
|
|
- **File Operations**:
|
|
- Upload with metadata
|
|
- Download (direct or presigned URLs)
|
|
- Delete
|
|
- Query with filters
|
|
- **Advanced Querying**:
|
|
- Filter by filename, file type, test name, test suite, test result
|
|
- Tag-based filtering
|
|
- Date range queries
|
|
- Pagination support
|
|
- **Auto-generated Documentation**: Swagger UI and ReDoc
|
|
|
|
### Feature Flags
|
|
|
|
#### Deployment Mode
|
|
Toggle between cloud and air-gapped environments:
|
|
|
|
```bash
|
|
# Air-gapped mode (default)
|
|
DEPLOYMENT_MODE=air-gapped
|
|
# Automatically uses MinIO for storage
|
|
|
|
# Cloud mode
|
|
DEPLOYMENT_MODE=cloud
|
|
# Automatically uses AWS S3 for storage
|
|
```
|
|
|
|
**Benefits**:
|
|
- Single codebase for both deployment scenarios
|
|
- Automatic backend configuration
|
|
- Easy environment switching
|
|
- No code changes required
|
|
|
|
### Test Utilities
|
|
|
|
#### Seed Data Generation
|
|
Generate realistic test data for development and testing:
|
|
|
|
**Quick Usage**:
|
|
```bash
|
|
# Generate 25 artifacts (default)
|
|
python seed.py
|
|
|
|
# Generate specific number
|
|
python seed.py 100
|
|
|
|
# Clear all data
|
|
python seed.py clear
|
|
```
|
|
|
|
**Advanced Usage**:
|
|
```bash
|
|
# Using the module directly
|
|
python -m utils.seed_data generate --count 50
|
|
|
|
# Clear all artifacts
|
|
python -m utils.seed_data clear
|
|
```
|
|
|
|
**Generated Data Includes**:
|
|
- CSV files with test results
|
|
- JSON configuration files
|
|
- Binary test data files
|
|
- PCAP network capture files
|
|
- Realistic metadata:
|
|
- Test names and suites
|
|
- Pass/fail/skip/error results
|
|
- Random tags
|
|
- Test configurations
|
|
- Version information
|
|
- Timestamps (last 30 days)
|
|
|
|
### Frontend (Angular 19)
|
|
|
|
**Modern Web Interface**:
|
|
- Built with Angular 19 standalone components
|
|
- Material Design theming and layout
|
|
- Responsive design
|
|
|
|
**Key Components**:
|
|
- **Artifact List**: Browse and manage artifacts with pagination
|
|
- **Upload Form**: Upload files with metadata input
|
|
- **Query Interface**: Advanced filtering and search
|
|
- **Detail View**: View full artifact information
|
|
- **Download/Delete**: Quick actions
|
|
|
|
**Features**:
|
|
- Real-time deployment mode indicator
|
|
- File type icons and badges
|
|
- Result status chips (pass/fail/skip/error)
|
|
- Responsive data tables
|
|
- Drag-and-drop file upload
|
|
|
|
### Deployment
|
|
|
|
#### Docker Support
|
|
- **Dockerized Application**: Single container for API
|
|
- **Docker Compose**: Complete stack (API + PostgreSQL + MinIO)
|
|
- **Multi-stage Builds**: Optimized image size
|
|
|
|
#### Kubernetes/Helm
|
|
- **Single Helm Chart**: Deploy entire stack
|
|
- **Configurable Values**: Resources, replicas, storage
|
|
- **Auto-scaling Support**: HPA for production
|
|
- **Health Checks**: Liveness and readiness probes
|
|
|
|
#### CI/CD
|
|
- **GitLab CI Pipeline**: Automated testing and deployment
|
|
- **Multi-environment**: Dev, staging, production
|
|
- **Manual Gates**: Control production deployments
|
|
- **Container Registry**: Automatic image building
|
|
|
|
### Security & Reliability
|
|
|
|
**Application**:
|
|
- Non-root container user
|
|
- Health check endpoints
|
|
- Structured logging
|
|
- Error handling and rollback
|
|
|
|
**Storage**:
|
|
- Presigned URLs for secure downloads
|
|
- UUID-based file naming (prevents conflicts)
|
|
- Automatic bucket creation
|
|
|
|
**Database**:
|
|
- Connection pooling
|
|
- Transaction management
|
|
- Indexed queries for performance
|
|
|
|
### Monitoring & Observability
|
|
|
|
**Health Checks**:
|
|
- `/health` endpoint for liveness
|
|
- Database connectivity check
|
|
- Storage backend verification
|
|
|
|
**Logging**:
|
|
- Structured logging format
|
|
- Configurable log levels
|
|
- Request/response logging
|
|
|
|
**Metrics** (Future):
|
|
- Prometheus endpoint
|
|
- Upload/download metrics
|
|
- Storage usage tracking
|
|
|
|
## Feature Comparison Matrix
|
|
|
|
| Feature | Cloud Mode | Air-Gapped Mode |
|
|
|---------|-----------|-----------------|
|
|
| Storage Backend | AWS S3 | MinIO |
|
|
| Database | RDS/Self-hosted PostgreSQL | Self-hosted PostgreSQL |
|
|
| Authentication | IAM/OAuth | Internal |
|
|
| Deployment | EKS/Cloud K8s | On-premise K8s |
|
|
| Cost Model | Pay-per-use | Fixed infrastructure |
|
|
| Scalability | Unlimited | Hardware-limited |
|
|
| Internet Required | Yes | No |
|
|
|
|
## Use Cases
|
|
|
|
### Test Automation
|
|
- Store test execution results (CSV)
|
|
- Archive test configurations (JSON)
|
|
- Track test history and trends
|
|
- Query by test suite, result, date
|
|
|
|
### Network Testing
|
|
- Store packet captures (PCAP)
|
|
- Associate captures with test runs
|
|
- Query by tags and metadata
|
|
- Download for analysis
|
|
|
|
### Build Artifacts
|
|
- Store binary test data
|
|
- Version control for test files
|
|
- Track across builds
|
|
- Query by version
|
|
|
|
### Compliance & Audit
|
|
- Immutable artifact storage
|
|
- Timestamp tracking
|
|
- Metadata for traceability
|
|
- Easy retrieval for audits
|
|
|
|
## Future Enhancements
|
|
|
|
### Planned Features
|
|
- [ ] Authentication & Authorization (OAuth, RBAC)
|
|
- [ ] File preview in UI
|
|
- [ ] Bulk upload API
|
|
- [ ] Advanced analytics dashboard
|
|
- [ ] Webhook notifications
|
|
- [ ] Full-text search (Elasticsearch)
|
|
- [ ] Automatic artifact retention policies
|
|
- [ ] Data export/import tools
|
|
- [ ] Performance metrics dashboard
|
|
- [ ] API rate limiting
|
|
|
|
### Under Consideration
|
|
- [ ] Multi-tenant support
|
|
- [ ] Artifact comparison tools
|
|
- [ ] Integration with CI/CD systems
|
|
- [ ] Automated report generation
|
|
- [ ] Machine learning for test prediction
|
|
- [ ] Distributed tracing
|
|
- [ ] Artifact deduplication
|
|
- [ ] Cost analysis dashboard
|