Orchard

Content-Addressable Storage System

Orchard is a centralized binary artifact storage system that provides content-addressable storage with automatic deduplication, flexible access control, and multi-format package support. Like an orchard that cultivates and distributes fruit, Orchard nurtures and distributes the products of software builds.

Features

Currently Implemented

• Content-Addressable Storage - Artifacts are stored and referenced by their SHA256 hash, ensuring deduplication and data integrity
• Grove/Tree/Fruit Hierarchy - Organized storage structure:
  • Grove - Top-level project container
  • Tree - Named package within a grove
  • Fruit - Specific artifact instance identified by SHA256
• Grafts (Tags/Versions) - Alias system for referencing artifacts by human-readable names (e.g., v1.0.0, latest, stable)
• S3-Compatible Backend - Uses MinIO (or any S3-compatible storage) for artifact storage
• PostgreSQL Metadata - Relational database for metadata, access control, and audit trails
• REST API - Full HTTP API for all operations
• Web UI - Browser-based interface for managing artifacts
• Docker Compose Setup - Easy local development environment

API Endpoints

Method	Endpoint	Description
GET	/	Web UI
GET	/health	Health check
GET	/api/v1/groves	List all groves
POST	/api/v1/groves	Create a new grove
GET	/api/v1/groves/:grove	Get grove details
GET	/api/v1/grove/:grove/trees	List trees in a grove
POST	/api/v1/grove/:grove/trees	Create a new tree
POST	/api/v1/grove/:grove/:tree/cultivate	Upload an artifact
GET	/api/v1/grove/:grove/:tree/+/:ref	Download an artifact
GET	/api/v1/grove/:grove/:tree/grafts	List all tags/versions
POST	/api/v1/grove/:grove/:tree/graft	Create a tag
GET	/api/v1/grove/:grove/:tree/consumers	List consumers of a tree
GET	/api/v1/fruit/:id	Get fruit metadata by hash

Reference Formats

When downloading artifacts, the :ref parameter supports multiple formats:

• latest - Tag name directly
• v1.0.0 - Version tag
• tag:stable - Explicit tag reference
• version:2024.1 - Version reference
• fruit:a3f5d8e12b4c6789... - Direct SHA256 hash reference

Quick Start

Prerequisites

• Docker and Docker Compose

Running Locally

# Start all services
docker-compose up -d

# View logs
docker-compose logs -f orchard-server

# Stop services
docker-compose down

Services

Service	Port	Description
orchard-server	8080	Main API server and Web UI
postgres	5432	PostgreSQL database
minio	9000	S3-compatible object storage
minio (console)	9001	MinIO web console
redis	6379	Cache (for future use)

Access Points

• Web UI: http://localhost:8080
• API: http://localhost:8080/api/v1
• MinIO Console: http://localhost:9001 (user: minioadmin, pass: minioadmin)

Usage Examples

Create a Grove

curl -X POST http://localhost:8080/api/v1/groves \
  -H "Content-Type: application/json" \
  -d '{"name": "my-project", "description": "My project artifacts", "is_public": true}'

Create a Tree

curl -X POST http://localhost:8080/api/v1/grove/my-project/trees \
  -H "Content-Type: application/json" \
  -d '{"name": "releases", "description": "Release builds"}'

Upload an Artifact (Cultivate)

curl -X POST http://localhost:8080/api/v1/grove/my-project/releases/cultivate \
  -F "file=@./build/app-v1.0.0.tar.gz" \
  -F "tag=v1.0.0"

Response:

{
  "fruit_id": "a3f5d8e12b4c67890abcdef1234567890abcdef1234567890abcdef12345678",
  "size": 1048576,
  "grove": "my-project",
  "tree": "releases",
  "tag": "v1.0.0"
}

Download an Artifact (Harvest)

# By tag
curl -O http://localhost:8080/api/v1/grove/my-project/releases/+/v1.0.0

# By fruit ID
curl -O http://localhost:8080/api/v1/grove/my-project/releases/+/fruit:a3f5d8e12b4c6789...

# Using the spec-compliant URL pattern
curl -O http://localhost:8080/grove/my-project/releases/+/latest

Create a Tag (Graft)

curl -X POST http://localhost:8080/api/v1/grove/my-project/releases/graft \
  -H "Content-Type: application/json" \
  -d '{"name": "stable", "fruit_id": "a3f5d8e12b4c6789..."}'

Search by Fruit ID

curl http://localhost:8080/api/v1/fruit/a3f5d8e12b4c67890abcdef1234567890abcdef1234567890abcdef12345678

Project Structure

orchard/
├── cmd/
│   └── orchard-server/
│       └── main.go              # Application entrypoint
├── internal/
│   ├── api/
│   │   ├── handlers.go          # HTTP request handlers
│   │   ├── router.go            # Route definitions
│   │   └── static/              # Web UI assets
│   │       ├── index.html
│   │       ├── style.css
│   │       └── app.js
│   ├── config/
│   │   └── config.go            # Configuration management
│   ├── models/
│   │   └── models.go            # Data structures
│   └── storage/
│       ├── database.go          # PostgreSQL operations
│       └── s3.go                # S3 storage operations
├── migrations/
│   └── 001_initial.sql          # Database schema
├── Dockerfile                   # Multi-stage Docker build
├── docker-compose.yml           # Local development stack
├── config.yaml                  # Default configuration
├── Makefile                     # Build automation
├── go.mod                       # Go module definition
└── go.sum                       # Dependency checksums

Configuration

Configuration can be provided via config.yaml or environment variables prefixed with ORCHARD_:

Environment Variable	Description	Default
ORCHARD_SERVER_HOST	Server bind address	0.0.0.0
ORCHARD_SERVER_PORT	Server port	8080
ORCHARD_DATABASE_HOST	PostgreSQL host	localhost
ORCHARD_DATABASE_PORT	PostgreSQL port	5432
ORCHARD_DATABASE_USER	PostgreSQL user	orchard
ORCHARD_DATABASE_PASSWORD	PostgreSQL password	-
ORCHARD_DATABASE_DBNAME	PostgreSQL database	orchard
ORCHARD_S3_ENDPOINT	S3 endpoint URL	-
ORCHARD_S3_REGION	S3 region	us-east-1
ORCHARD_S3_BUCKET	S3 bucket name	orchard-artifacts
ORCHARD_S3_ACCESS_KEY_ID	S3 access key	-
ORCHARD_S3_SECRET_ACCESS_KEY	S3 secret key	-

Database Schema

Core Tables

• groves - Top-level project containers
• trees - Packages within groves
• fruits - Content-addressable artifacts (SHA256)
• grafts - Tags/aliases pointing to fruits
• graft_history - Audit trail for tag changes
• harvests - Upload event records
• consumers - Dependency tracking
• access_permissions - Grove-level access control
• api_keys - Programmatic access tokens
• audit_logs - Immutable operation logs

Terminology

Orchard Term	Traditional Term	Description
Grove	Project	Top-level organizational unit
Tree	Package	Named collection of related artifacts
Fruit	Instance	Specific content identified by SHA256
Seed	Dependency	Required package specification
Harvest	Download/Fetch	Retrieve dependencies
Cultivate	Upload/Publish	Store new artifact
Graft	Alias/Tag	Alternative name for content
Prune	Clean/GC	Remove unused local cache

Future Work

The following features from the specification are planned but not yet implemented:

• CLI tool (orchard command)
• orchard.ensure file parsing
• Lock file generation (orchard.lock)
• Export/Import for air-gapped systems
• Consumer notification (pollination)
• Automated update propagation
• OIDC/SAML authentication
• API key management
• Package format detection
• Multipart upload for large files
• Redis caching layer

License

Internal use only.