bitforge/orchard
3
0
Fork 0
Code Projects Wiki Activity

Add comprehensive README documentation

Browse Source 
- Project overview and features
- Quick start guide
- API endpoint reference
- Usage examples with curl
- Project structure
- Configuration options
- Database schema overview
- Terminology glossary
- Future work roadmap
This commit is contained in:
Mondo Diaz
2025-12-04 10:17:18 -06:00
parent f8e9650de3
commit a0e350ee7f
1 changed files with 247 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

247
README.md Normal file
View File

@@ -0,0 +1,247 @@
# Orchard
**Content-Addressable Storage System**
Orchard is a centralized binary artifact storage system that provides content-addressable storage with automatic deduplication, flexible access control, and multi-format package support. Like an orchard that cultivates and distributes fruit, Orchard nurtures and distributes the products of software builds.
## Features
### Currently Implemented
- **Content-Addressable Storage** - Artifacts are stored and referenced by their SHA256 hash, ensuring deduplication and data integrity
- **Grove/Tree/Fruit Hierarchy** - Organized storage structure:
- **Grove** - Top-level project container
- **Tree** - Named package within a grove
- **Fruit** - Specific artifact instance identified by SHA256
- **Grafts (Tags/Versions)** - Alias system for referencing artifacts by human-readable names (e.g., `v1.0.0`, `latest`, `stable`)
- **S3-Compatible Backend** - Uses MinIO (or any S3-compatible storage) for artifact storage
- **PostgreSQL Metadata** - Relational database for metadata, access control, and audit trails
- **REST API** - Full HTTP API for all operations
- **Web UI** - Browser-based interface for managing artifacts
- **Docker Compose Setup** - Easy local development environment
### API Endpoints
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/` | Web UI |
| `GET` | `/health` | Health check |
| `GET` | `/api/v1/groves` | List all groves |
| `POST` | `/api/v1/groves` | Create a new grove |
| `GET` | `/api/v1/groves/:grove` | Get grove details |
| `GET` | `/api/v1/grove/:grove/trees` | List trees in a grove |
| `POST` | `/api/v1/grove/:grove/trees` | Create a new tree |
| `POST` | `/api/v1/grove/:grove/:tree/cultivate` | Upload an artifact |
| `GET` | `/api/v1/grove/:grove/:tree/+/:ref` | Download an artifact |
| `GET` | `/api/v1/grove/:grove/:tree/grafts` | List all tags/versions |
| `POST` | `/api/v1/grove/:grove/:tree/graft` | Create a tag |
| `GET` | `/api/v1/grove/:grove/:tree/consumers` | List consumers of a tree |
| `GET` | `/api/v1/fruit/:id` | Get fruit metadata by hash |
### Reference Formats
When downloading artifacts, the `:ref` parameter supports multiple formats:
- `latest` - Tag name directly
- `v1.0.0` - Version tag
- `tag:stable` - Explicit tag reference
- `version:2024.1` - Version reference
- `fruit:a3f5d8e12b4c6789...` - Direct SHA256 hash reference
## Quick Start
### Prerequisites
- Docker and Docker Compose
### Running Locally
```bash
# Start all services
docker-compose up -d
# View logs
docker-compose logs -f orchard-server
# Stop services
docker-compose down
```
### Services
| Service | Port | Description |
|---------|------|-------------|
| orchard-server | 8080 | Main API server and Web UI |
| postgres | 5432 | PostgreSQL database |
| minio | 9000 | S3-compatible object storage |
| minio (console) | 9001 | MinIO web console |
| redis | 6379 | Cache (for future use) |
### Access Points
- **Web UI**: http://localhost:8080
- **API**: http://localhost:8080/api/v1
- **MinIO Console**: http://localhost:9001 (user: `minioadmin`, pass: `minioadmin`)
## Usage Examples
### Create a Grove
```bash
curl -X POST http://localhost:8080/api/v1/groves \
-H "Content-Type: application/json" \
-d '{"name": "my-project", "description": "My project artifacts", "is_public": true}'
```
### Create a Tree
```bash
curl -X POST http://localhost:8080/api/v1/grove/my-project/trees \
-H "Content-Type: application/json" \
-d '{"name": "releases", "description": "Release builds"}'
```
### Upload an Artifact (Cultivate)
```bash
curl -X POST http://localhost:8080/api/v1/grove/my-project/releases/cultivate \
-F "file=@./build/app-v1.0.0.tar.gz" \
-F "tag=v1.0.0"
```
Response:
```json
{
"fruit_id": "a3f5d8e12b4c67890abcdef1234567890abcdef1234567890abcdef12345678",
"size": 1048576,
"grove": "my-project",
"tree": "releases",
"tag": "v1.0.0"
}
```
### Download an Artifact (Harvest)
```bash
# By tag
curl -O http://localhost:8080/api/v1/grove/my-project/releases/+/v1.0.0
# By fruit ID
curl -O http://localhost:8080/api/v1/grove/my-project/releases/+/fruit:a3f5d8e12b4c6789...
# Using the spec-compliant URL pattern
curl -O http://localhost:8080/grove/my-project/releases/+/latest
```
### Create a Tag (Graft)
```bash
curl -X POST http://localhost:8080/api/v1/grove/my-project/releases/graft \
-H "Content-Type: application/json" \
-d '{"name": "stable", "fruit_id": "a3f5d8e12b4c6789..."}'
```
### Search by Fruit ID
```bash
curl http://localhost:8080/api/v1/fruit/a3f5d8e12b4c67890abcdef1234567890abcdef1234567890abcdef12345678
```
## Project Structure
```
orchard/
├── cmd/
│ └── orchard-server/
│ └── main.go # Application entrypoint
├── internal/
│ ├── api/
│ │ ├── handlers.go # HTTP request handlers
│ │ ├── router.go # Route definitions
│ │ └── static/ # Web UI assets
│ │ ├── index.html
│ │ ├── style.css
│ │ └── app.js
│ ├── config/
│ │ └── config.go # Configuration management
│ ├── models/
│ │ └── models.go # Data structures
│ └── storage/
│ ├── database.go # PostgreSQL operations
│ └── s3.go # S3 storage operations
├── migrations/
│ └── 001_initial.sql # Database schema
├── Dockerfile # Multi-stage Docker build
├── docker-compose.yml # Local development stack
├── config.yaml # Default configuration
├── Makefile # Build automation
├── go.mod # Go module definition
└── go.sum # Dependency checksums
```
## Configuration
Configuration can be provided via `config.yaml` or environment variables prefixed with `ORCHARD_`:
| Environment Variable | Description | Default |
|---------------------|-------------|---------|
| `ORCHARD_SERVER_HOST` | Server bind address | `0.0.0.0` |
| `ORCHARD_SERVER_PORT` | Server port | `8080` |
| `ORCHARD_DATABASE_HOST` | PostgreSQL host | `localhost` |
| `ORCHARD_DATABASE_PORT` | PostgreSQL port | `5432` |
| `ORCHARD_DATABASE_USER` | PostgreSQL user | `orchard` |
| `ORCHARD_DATABASE_PASSWORD` | PostgreSQL password | - |
| `ORCHARD_DATABASE_DBNAME` | PostgreSQL database | `orchard` |
| `ORCHARD_S3_ENDPOINT` | S3 endpoint URL | - |
| `ORCHARD_S3_REGION` | S3 region | `us-east-1` |
| `ORCHARD_S3_BUCKET` | S3 bucket name | `orchard-artifacts` |
| `ORCHARD_S3_ACCESS_KEY_ID` | S3 access key | - |
| `ORCHARD_S3_SECRET_ACCESS_KEY` | S3 secret key | - |
## Database Schema
### Core Tables
- **groves** - Top-level project containers
- **trees** - Packages within groves
- **fruits** - Content-addressable artifacts (SHA256)
- **grafts** - Tags/aliases pointing to fruits
- **graft_history** - Audit trail for tag changes
- **harvests** - Upload event records
- **consumers** - Dependency tracking
- **access_permissions** - Grove-level access control
- **api_keys** - Programmatic access tokens
- **audit_logs** - Immutable operation logs
## Terminology
| Orchard Term | Traditional Term | Description |
|--------------|------------------|-------------|
| Grove | Project | Top-level organizational unit |
| Tree | Package | Named collection of related artifacts |
| Fruit | Instance | Specific content identified by SHA256 |
| Seed | Dependency | Required package specification |
| Harvest | Download/Fetch | Retrieve dependencies |
| Cultivate | Upload/Publish | Store new artifact |
| Graft | Alias/Tag | Alternative name for content |
| Prune | Clean/GC | Remove unused local cache |
## Future Work
The following features from the specification are planned but not yet implemented:
- [ ] CLI tool (`orchard` command)
- [ ] `orchard.ensure` file parsing
- [ ] Lock file generation (`orchard.lock`)
- [ ] Export/Import for air-gapped systems
- [ ] Consumer notification (pollination)
- [ ] Automated update propagation
- [ ] OIDC/SAML authentication
- [ ] API key management
- [ ] Package format detection
- [ ] Multipart upload for large files
- [ ] Redis caching layer
## License
Internal use only.