orchard/README.md

# Orchard

**Content-Addressable Storage System**

Orchard is a centralized binary artifact storage system that provides content-addressable storage with automatic deduplication, flexible access control, and multi-format package support. Like an orchard that cultivates and distributes fruit, Orchard nurtures and distributes the products of software builds.

## Features

### Currently Implemented

- **Content-Addressable Storage** - Artifacts are stored and referenced by their SHA256 hash, ensuring deduplication and data integrity
- **Grove/Tree/Fruit Hierarchy** - Organized storage structure:
  - **Grove** - Top-level project container
  - **Tree** - Named package within a grove
  - **Fruit** - Specific artifact instance identified by SHA256
- **Grafts (Tags/Versions)** - Alias system for referencing artifacts by human-readable names (e.g., `v1.0.0`, `latest`, `stable`)
- **S3-Compatible Backend** - Uses MinIO (or any S3-compatible storage) for artifact storage
- **PostgreSQL Metadata** - Relational database for metadata, access control, and audit trails
- **REST API** - Full HTTP API for all operations
- **Web UI** - Browser-based interface for managing artifacts
- **Docker Compose Setup** - Easy local development environment

### API Endpoints

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/` | Web UI |
| `GET` | `/health` | Health check |
| `GET` | `/api/v1/groves` | List all groves |
| `POST` | `/api/v1/groves` | Create a new grove |
| `GET` | `/api/v1/groves/:grove` | Get grove details |
| `GET` | `/api/v1/grove/:grove/trees` | List trees in a grove |
| `POST` | `/api/v1/grove/:grove/trees` | Create a new tree |
| `POST` | `/api/v1/grove/:grove/:tree/cultivate` | Upload an artifact |
| `GET` | `/api/v1/grove/:grove/:tree/+/:ref` | Download an artifact |
| `GET` | `/api/v1/grove/:grove/:tree/grafts` | List all tags/versions |
| `POST` | `/api/v1/grove/:grove/:tree/graft` | Create a tag |
| `GET` | `/api/v1/grove/:grove/:tree/consumers` | List consumers of a tree |
| `GET` | `/api/v1/fruit/:id` | Get fruit metadata by hash |

### Reference Formats

When downloading artifacts, the `:ref` parameter supports multiple formats:

- `latest` - Tag name directly
- `v1.0.0` - Version tag
- `tag:stable` - Explicit tag reference
- `version:2024.1` - Version reference
- `fruit:a3f5d8e12b4c6789...` - Direct SHA256 hash reference

## Quick Start

### Prerequisites

- Docker and Docker Compose

### Running Locally

```bash
# Start all services
docker-compose up -d

# View logs
docker-compose logs -f orchard-server

# Stop services
docker-compose down
```

### Services

| Service | Port | Description |
|---------|------|-------------|
| orchard-server | 8080 | Main API server and Web UI |
| postgres | 5432 | PostgreSQL database |
| minio | 9000 | S3-compatible object storage |
| minio (console) | 9001 | MinIO web console |
| redis | 6379 | Cache (for future use) |

### Access Points

- **Web UI**: http://localhost:8080
- **API**: http://localhost:8080/api/v1
- **MinIO Console**: http://localhost:9001 (user: `minioadmin`, pass: `minioadmin`)

## Usage Examples

### Create a Grove

```bash
curl -X POST http://localhost:8080/api/v1/groves \
  -H "Content-Type: application/json" \
  -d '{"name": "my-project", "description": "My project artifacts", "is_public": true}'
```

### Create a Tree

```bash
curl -X POST http://localhost:8080/api/v1/grove/my-project/trees \
  -H "Content-Type: application/json" \
  -d '{"name": "releases", "description": "Release builds"}'
```

### Upload an Artifact (Cultivate)

```bash
curl -X POST http://localhost:8080/api/v1/grove/my-project/releases/cultivate \
  -F "file=@./build/app-v1.0.0.tar.gz" \
  -F "tag=v1.0.0"
```

Response:
```json
{
  "fruit_id": "a3f5d8e12b4c67890abcdef1234567890abcdef1234567890abcdef12345678",
  "size": 1048576,
  "grove": "my-project",
  "tree": "releases",
  "tag": "v1.0.0"
}
```

### Download an Artifact (Harvest)

```bash
# By tag
curl -O http://localhost:8080/api/v1/grove/my-project/releases/+/v1.0.0

# By fruit ID
curl -O http://localhost:8080/api/v1/grove/my-project/releases/+/fruit:a3f5d8e12b4c6789...

# Using the spec-compliant URL pattern
curl -O http://localhost:8080/grove/my-project/releases/+/latest
```

### Create a Tag (Graft)

```bash
curl -X POST http://localhost:8080/api/v1/grove/my-project/releases/graft \
  -H "Content-Type: application/json" \
  -d '{"name": "stable", "fruit_id": "a3f5d8e12b4c6789..."}'
```

### Search by Fruit ID

```bash
curl http://localhost:8080/api/v1/fruit/a3f5d8e12b4c67890abcdef1234567890abcdef1234567890abcdef12345678
```

## Project Structure

```
orchard/
├── cmd/
│   └── orchard-server/
│       └── main.go              # Application entrypoint
├── internal/
│   ├── api/
│   │   ├── handlers.go          # HTTP request handlers
│   │   ├── router.go            # Route definitions
│   │   └── static/              # Web UI assets
│   │       ├── index.html
│   │       ├── style.css
│   │       └── app.js
│   ├── config/
│   │   └── config.go            # Configuration management
│   ├── models/
│   │   └── models.go            # Data structures
│   └── storage/
│       ├── database.go          # PostgreSQL operations
│       └── s3.go                # S3 storage operations
├── migrations/
│   └── 001_initial.sql          # Database schema
├── Dockerfile                   # Multi-stage Docker build
├── docker-compose.yml           # Local development stack
├── config.yaml                  # Default configuration
├── Makefile                     # Build automation
├── go.mod                       # Go module definition
└── go.sum                       # Dependency checksums
```

## Configuration

Configuration can be provided via `config.yaml` or environment variables prefixed with `ORCHARD_`:

| Environment Variable | Description | Default |
|---------------------|-------------|---------|
| `ORCHARD_SERVER_HOST` | Server bind address | `0.0.0.0` |
| `ORCHARD_SERVER_PORT` | Server port | `8080` |
| `ORCHARD_DATABASE_HOST` | PostgreSQL host | `localhost` |
| `ORCHARD_DATABASE_PORT` | PostgreSQL port | `5432` |
| `ORCHARD_DATABASE_USER` | PostgreSQL user | `orchard` |
| `ORCHARD_DATABASE_PASSWORD` | PostgreSQL password | - |
| `ORCHARD_DATABASE_DBNAME` | PostgreSQL database | `orchard` |
| `ORCHARD_S3_ENDPOINT` | S3 endpoint URL | - |
| `ORCHARD_S3_REGION` | S3 region | `us-east-1` |
| `ORCHARD_S3_BUCKET` | S3 bucket name | `orchard-artifacts` |
| `ORCHARD_S3_ACCESS_KEY_ID` | S3 access key | - |
| `ORCHARD_S3_SECRET_ACCESS_KEY` | S3 secret key | - |

## Database Schema

### Core Tables

- **groves** - Top-level project containers
- **trees** - Packages within groves
- **fruits** - Content-addressable artifacts (SHA256)
- **grafts** - Tags/aliases pointing to fruits
- **graft_history** - Audit trail for tag changes
- **harvests** - Upload event records
- **consumers** - Dependency tracking
- **access_permissions** - Grove-level access control
- **api_keys** - Programmatic access tokens
- **audit_logs** - Immutable operation logs

## Terminology

| Orchard Term | Traditional Term | Description |
|--------------|------------------|-------------|
| Grove | Project | Top-level organizational unit |
| Tree | Package | Named collection of related artifacts |
| Fruit | Instance | Specific content identified by SHA256 |
| Seed | Dependency | Required package specification |
| Harvest | Download/Fetch | Retrieve dependencies |
| Cultivate | Upload/Publish | Store new artifact |
| Graft | Alias/Tag | Alternative name for content |
| Prune | Clean/GC | Remove unused local cache |

## Future Work

The following features from the specification are planned but not yet implemented:

- [ ] CLI tool (`orchard` command)
- [ ] `orchard.ensure` file parsing
- [ ] Lock file generation (`orchard.lock`)
- [ ] Export/Import for air-gapped systems
- [ ] Consumer notification (pollination)
- [ ] Automated update propagation
- [ ] OIDC/SAML authentication
- [ ] API key management
- [ ] Package format detection
- [ ] Multipart upload for large files
- [ ] Redis caching layer

## License

Internal use only.