orchard/README.md

Orchard

Content-Addressable Storage System

Orchard is a centralized binary artifact storage system that provides content-addressable storage with automatic deduplication, flexible access control, and multi-format package support. Like an orchard that cultivates and distributes fruit, Orchard nurtures and distributes the products of software builds.

Tech Stack

• Backend: Python 3.12 + FastAPI
• Frontend: React 18 + TypeScript + Vite
• Database: PostgreSQL 16
• Object Storage: MinIO (S3-compatible)
• Cache: Redis (for future use)

Features

Currently Implemented

• Content-Addressable Storage - Artifacts are stored and referenced by their SHA256 hash, ensuring deduplication and data integrity
• Grove/Tree/Fruit Hierarchy - Organized storage structure:
  • Grove - Top-level project container
  • Tree - Named package within a grove
  • Fruit - Specific artifact instance identified by SHA256
• Grafts (Tags/Versions) - Alias system for referencing artifacts by human-readable names (e.g., v1.0.0, latest, stable)
• S3-Compatible Backend - Uses MinIO (or any S3-compatible storage) for artifact storage
• PostgreSQL Metadata - Relational database for metadata, access control, and audit trails
• REST API - Full HTTP API for all operations
• Web UI - React-based interface for managing artifacts
• Docker Compose Setup - Easy local development environment
• Helm Chart - Kubernetes deployment with PostgreSQL, MinIO, and Redis subcharts

API Endpoints

Method	Endpoint	Description
GET	/	Web UI
GET	/health	Health check
GET	/api/v1/groves	List all groves
POST	/api/v1/groves	Create a new grove
GET	/api/v1/groves/:grove	Get grove details
GET	/api/v1/grove/:grove/trees	List trees in a grove
POST	/api/v1/grove/:grove/trees	Create a new tree
POST	/api/v1/grove/:grove/:tree/cultivate	Upload an artifact
GET	/api/v1/grove/:grove/:tree/+/:ref	Download an artifact
GET	/api/v1/grove/:grove/:tree/grafts	List all tags/versions
POST	/api/v1/grove/:grove/:tree/graft	Create a tag
GET	/api/v1/grove/:grove/:tree/consumers	List consumers of a tree
GET	/api/v1/fruit/:id	Get fruit metadata by hash

Reference Formats

When downloading artifacts, the :ref parameter supports multiple formats:

• latest - Tag name directly
• v1.0.0 - Version tag
• tag:stable - Explicit tag reference
• version:2024.1 - Version reference
• fruit:a3f5d8e12b4c6789... - Direct SHA256 hash reference

Quick Start

Prerequisites

• Docker and Docker Compose

Running Locally

# Start all services
docker-compose up -d

# View logs
docker-compose logs -f orchard-server

# Stop services
docker-compose down

Services

Service	Port	Description
orchard-server	8080	Main API server and Web UI
postgres	5432	PostgreSQL database
minio	9000	S3-compatible object storage
minio (console)	9001	MinIO web console
redis	6379	Cache (for future use)

Access Points

• Web UI: http://localhost:8080
• API: http://localhost:8080/api/v1
• API Docs: http://localhost:8080/docs
• MinIO Console: http://localhost:9001 (user: minioadmin, pass: minioadmin)

Development

Backend (FastAPI)

cd backend
pip install -r requirements.txt
uvicorn app.main:app --reload --port 8080

Frontend (React)

cd frontend
npm install
npm run dev

The frontend dev server proxies API requests to localhost:8080.

Usage Examples

Create a Grove

curl -X POST http://localhost:8080/api/v1/groves \
  -H "Content-Type: application/json" \
  -d '{"name": "my-project", "description": "My project artifacts", "is_public": true}'

Create a Tree

curl -X POST http://localhost:8080/api/v1/grove/my-project/trees \
  -H "Content-Type: application/json" \
  -d '{"name": "releases", "description": "Release builds"}'

Upload an Artifact (Cultivate)

curl -X POST http://localhost:8080/api/v1/grove/my-project/releases/cultivate \
  -F "file=@./build/app-v1.0.0.tar.gz" \
  -F "tag=v1.0.0"

Response:

{
  "fruit_id": "a3f5d8e12b4c67890abcdef1234567890abcdef1234567890abcdef12345678",
  "size": 1048576,
  "grove": "my-project",
  "tree": "releases",
  "tag": "v1.0.0"
}

Download an Artifact (Harvest)

# By tag
curl -O http://localhost:8080/api/v1/grove/my-project/releases/+/v1.0.0

# By fruit ID
curl -O http://localhost:8080/api/v1/grove/my-project/releases/+/fruit:a3f5d8e12b4c6789...

# Using the spec-compliant URL pattern
curl -O http://localhost:8080/grove/my-project/releases/+/latest

Create a Tag (Graft)

curl -X POST http://localhost:8080/api/v1/grove/my-project/releases/graft \
  -H "Content-Type: application/json" \
  -d '{"name": "stable", "fruit_id": "a3f5d8e12b4c6789..."}'

Search by Fruit ID

curl http://localhost:8080/api/v1/fruit/a3f5d8e12b4c67890abcdef1234567890abcdef1234567890abcdef12345678

Project Structure

orchard/
├── backend/
│   ├── app/
│   │   ├── __init__.py
│   │   ├── config.py           # Pydantic settings
│   │   ├── database.py         # SQLAlchemy setup
│   │   ├── main.py             # FastAPI application
│   │   ├── models.py           # SQLAlchemy models
│   │   ├── routes.py           # API endpoints
│   │   ├── schemas.py          # Pydantic schemas
│   │   └── storage.py          # S3 storage layer
│   └── requirements.txt
├── frontend/
│   ├── src/
│   │   ├── components/         # React components
│   │   ├── pages/              # Page components
│   │   ├── api.ts              # API client
│   │   ├── types.ts            # TypeScript types
│   │   ├── App.tsx
│   │   └── main.tsx
│   ├── index.html
│   ├── package.json
│   ├── tsconfig.json
│   └── vite.config.ts
├── helm/
│   └── orchard/                # Helm chart
├── migrations/
│   └── 001_initial.sql         # Database schema
├── Dockerfile                  # Multi-stage build (Node + Python)
├── docker-compose.yml          # Local development stack
└── .gitlab-ci.yml              # CI/CD pipeline

Configuration

Configuration is provided via environment variables prefixed with ORCHARD_:

Environment Variable	Description	Default
ORCHARD_SERVER_HOST	Server bind address	0.0.0.0
ORCHARD_SERVER_PORT	Server port	8080
ORCHARD_DATABASE_HOST	PostgreSQL host	localhost
ORCHARD_DATABASE_PORT	PostgreSQL port	5432
ORCHARD_DATABASE_USER	PostgreSQL user	orchard
ORCHARD_DATABASE_PASSWORD	PostgreSQL password	-
ORCHARD_DATABASE_DBNAME	PostgreSQL database	orchard
ORCHARD_S3_ENDPOINT	S3 endpoint URL	-
ORCHARD_S3_REGION	S3 region	us-east-1
ORCHARD_S3_BUCKET	S3 bucket name	orchard-artifacts
ORCHARD_S3_ACCESS_KEY_ID	S3 access key	-
ORCHARD_S3_SECRET_ACCESS_KEY	S3 secret key	-

Kubernetes Deployment

Using Helm

# Add Bitnami repo for dependencies
helm repo add bitnami https://charts.bitnami.com/bitnami

# Update dependencies
cd helm/orchard
helm dependency update

# Install
helm install orchard ./helm/orchard -n orchard --create-namespace

# Install with custom values
helm install orchard ./helm/orchard -f my-values.yaml

See helm/orchard/values.yaml for all configuration options.

Database Schema

Core Tables

• groves - Top-level project containers
• trees - Packages within groves
• fruits - Content-addressable artifacts (SHA256)
• grafts - Tags/aliases pointing to fruits
• graft_history - Audit trail for tag changes
• harvests - Upload event records
• consumers - Dependency tracking
• access_permissions - Grove-level access control
• api_keys - Programmatic access tokens
• audit_logs - Immutable operation logs

Terminology

Orchard Term	Traditional Term	Description
Grove	Project	Top-level organizational unit
Tree	Package	Named collection of related artifacts
Fruit	Instance	Specific content identified by SHA256
Seed	Dependency	Required package specification
Harvest	Download/Fetch	Retrieve dependencies
Cultivate	Upload/Publish	Store new artifact
Graft	Alias/Tag	Alternative name for content
Prune	Clean/GC	Remove unused local cache

Future Work

The following features from the specification are planned but not yet implemented:

• CLI tool (orchard command)
• orchard.ensure file parsing
• Lock file generation (orchard.lock)
• Export/Import for air-gapped systems
• Consumer notification (pollination)
• Automated update propagation
• OIDC/SAML authentication
• API key management
• Package format detection
• Multipart upload for large files
• Redis caching layer

License

Internal use only.