orchard/README.md

# Orchard

**Content-Addressable Storage System**

Orchard is a centralized binary artifact storage system that provides content-addressable storage with automatic deduplication, flexible access control, and multi-format package support. Like an orchard that cultivates and distributes fruit, Orchard nurtures and distributes the products of software builds.

## Tech Stack

- **Backend**: Python 3.12 + FastAPI
- **Frontend**: React 18 + TypeScript + Vite
- **Database**: PostgreSQL 16
- **Object Storage**: MinIO (S3-compatible)
- **Cache**: Redis (for future use)

## Features

### Currently Implemented

- **Content-Addressable Storage** - Artifacts are stored and referenced by their SHA256 hash, ensuring deduplication and data integrity
- **Grove/Tree/Fruit Hierarchy** - Organized storage structure:
  - **Grove** - Top-level project container
  - **Tree** - Named package within a grove
  - **Fruit** - Specific artifact instance identified by SHA256
- **Grafts (Tags/Versions)** - Alias system for referencing artifacts by human-readable names (e.g., `v1.0.0`, `latest`, `stable`)
- **S3-Compatible Backend** - Uses MinIO (or any S3-compatible storage) for artifact storage
- **PostgreSQL Metadata** - Relational database for metadata, access control, and audit trails
- **REST API** - Full HTTP API for all operations
- **Web UI** - React-based interface for managing artifacts
- **Docker Compose Setup** - Easy local development environment
- **Helm Chart** - Kubernetes deployment with PostgreSQL, MinIO, and Redis subcharts

### API Endpoints

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/` | Web UI |
| `GET` | `/health` | Health check |
| `GET` | `/api/v1/groves` | List all groves |
| `POST` | `/api/v1/groves` | Create a new grove |
| `GET` | `/api/v1/groves/:grove` | Get grove details |
| `GET` | `/api/v1/grove/:grove/trees` | List trees in a grove |
| `POST` | `/api/v1/grove/:grove/trees` | Create a new tree |
| `POST` | `/api/v1/grove/:grove/:tree/cultivate` | Upload an artifact |
| `GET` | `/api/v1/grove/:grove/:tree/+/:ref` | Download an artifact |
| `GET` | `/api/v1/grove/:grove/:tree/grafts` | List all tags/versions |
| `POST` | `/api/v1/grove/:grove/:tree/graft` | Create a tag |
| `GET` | `/api/v1/grove/:grove/:tree/consumers` | List consumers of a tree |
| `GET` | `/api/v1/fruit/:id` | Get fruit metadata by hash |

### Reference Formats

When downloading artifacts, the `:ref` parameter supports multiple formats:

- `latest` - Tag name directly
- `v1.0.0` - Version tag
- `tag:stable` - Explicit tag reference
- `version:2024.1` - Version reference
- `fruit:a3f5d8e12b4c6789...` - Direct SHA256 hash reference

## Quick Start

### Prerequisites

- Docker and Docker Compose

### Running Locally

```bash
# Start all services
docker-compose up -d

# View logs
docker-compose logs -f orchard-server

# Stop services
docker-compose down
```

### Services

| Service | Port | Description |
|---------|------|-------------|
| orchard-server | 8080 | Main API server and Web UI |
| postgres | 5432 | PostgreSQL database |
| minio | 9000 | S3-compatible object storage |
| minio (console) | 9001 | MinIO web console |
| redis | 6379 | Cache (for future use) |

### Access Points

- **Web UI**: http://localhost:8080
- **API**: http://localhost:8080/api/v1
- **API Docs**: http://localhost:8080/docs
- **MinIO Console**: http://localhost:9001 (user: `minioadmin`, pass: `minioadmin`)

## Development

### Backend (FastAPI)

```bash
cd backend
pip install -r requirements.txt
uvicorn app.main:app --reload --port 8080
```

### Frontend (React)

```bash
cd frontend
npm install
npm run dev
```

The frontend dev server proxies API requests to `localhost:8080`.

## Usage Examples

### Create a Grove

```bash
curl -X POST http://localhost:8080/api/v1/groves \
  -H "Content-Type: application/json" \
  -d '{"name": "my-project", "description": "My project artifacts", "is_public": true}'
```

### Create a Tree

```bash
curl -X POST http://localhost:8080/api/v1/grove/my-project/trees \
  -H "Content-Type: application/json" \
  -d '{"name": "releases", "description": "Release builds"}'
```

### Upload an Artifact (Cultivate)

```bash
curl -X POST http://localhost:8080/api/v1/grove/my-project/releases/cultivate \
  -F "file=@./build/app-v1.0.0.tar.gz" \
  -F "tag=v1.0.0"
```

Response:
```json
{
  "fruit_id": "a3f5d8e12b4c67890abcdef1234567890abcdef1234567890abcdef12345678",
  "size": 1048576,
  "grove": "my-project",
  "tree": "releases",
  "tag": "v1.0.0"
}
```

### Download an Artifact (Harvest)

```bash
# By tag
curl -O http://localhost:8080/api/v1/grove/my-project/releases/+/v1.0.0

# By fruit ID
curl -O http://localhost:8080/api/v1/grove/my-project/releases/+/fruit:a3f5d8e12b4c6789...

# Using the spec-compliant URL pattern
curl -O http://localhost:8080/grove/my-project/releases/+/latest
```

### Create a Tag (Graft)

```bash
curl -X POST http://localhost:8080/api/v1/grove/my-project/releases/graft \
  -H "Content-Type: application/json" \
  -d '{"name": "stable", "fruit_id": "a3f5d8e12b4c6789..."}'
```

### Search by Fruit ID

```bash
curl http://localhost:8080/api/v1/fruit/a3f5d8e12b4c67890abcdef1234567890abcdef1234567890abcdef12345678
```

## Project Structure

```
orchard/
├── backend/
│   ├── app/
│   │   ├── __init__.py
│   │   ├── config.py           # Pydantic settings
│   │   ├── database.py         # SQLAlchemy setup
│   │   ├── main.py             # FastAPI application
│   │   ├── models.py           # SQLAlchemy models
│   │   ├── routes.py           # API endpoints
│   │   ├── schemas.py          # Pydantic schemas
│   │   └── storage.py          # S3 storage layer
│   └── requirements.txt
├── frontend/
│   ├── src/
│   │   ├── components/         # React components
│   │   ├── pages/              # Page components
│   │   ├── api.ts              # API client
│   │   ├── types.ts            # TypeScript types
│   │   ├── App.tsx
│   │   └── main.tsx
│   ├── index.html
│   ├── package.json
│   ├── tsconfig.json
│   └── vite.config.ts
├── helm/
│   └── orchard/                # Helm chart
├── migrations/
│   └── 001_initial.sql         # Database schema
├── Dockerfile                  # Multi-stage build (Node + Python)
├── docker-compose.yml          # Local development stack
└── .gitlab-ci.yml              # CI/CD pipeline
```

## Configuration

Configuration is provided via environment variables prefixed with `ORCHARD_`:

| Environment Variable | Description | Default |
|---------------------|-------------|---------|
| `ORCHARD_SERVER_HOST` | Server bind address | `0.0.0.0` |
| `ORCHARD_SERVER_PORT` | Server port | `8080` |
| `ORCHARD_DATABASE_HOST` | PostgreSQL host | `localhost` |
| `ORCHARD_DATABASE_PORT` | PostgreSQL port | `5432` |
| `ORCHARD_DATABASE_USER` | PostgreSQL user | `orchard` |
| `ORCHARD_DATABASE_PASSWORD` | PostgreSQL password | - |
| `ORCHARD_DATABASE_DBNAME` | PostgreSQL database | `orchard` |
| `ORCHARD_S3_ENDPOINT` | S3 endpoint URL | - |
| `ORCHARD_S3_REGION` | S3 region | `us-east-1` |
| `ORCHARD_S3_BUCKET` | S3 bucket name | `orchard-artifacts` |
| `ORCHARD_S3_ACCESS_KEY_ID` | S3 access key | - |
| `ORCHARD_S3_SECRET_ACCESS_KEY` | S3 secret key | - |

## Kubernetes Deployment

### Using Helm

```bash
# Add Bitnami repo for dependencies
helm repo add bitnami https://charts.bitnami.com/bitnami

# Update dependencies
cd helm/orchard
helm dependency update

# Install
helm install orchard ./helm/orchard -n orchard --create-namespace

# Install with custom values
helm install orchard ./helm/orchard -f my-values.yaml
```

See `helm/orchard/values.yaml` for all configuration options.

## Database Schema

### Core Tables

- **groves** - Top-level project containers
- **trees** - Packages within groves
- **fruits** - Content-addressable artifacts (SHA256)
- **grafts** - Tags/aliases pointing to fruits
- **graft_history** - Audit trail for tag changes
- **harvests** - Upload event records
- **consumers** - Dependency tracking
- **access_permissions** - Grove-level access control
- **api_keys** - Programmatic access tokens
- **audit_logs** - Immutable operation logs

## Terminology

| Orchard Term | Traditional Term | Description |
|--------------|------------------|-------------|
| Grove | Project | Top-level organizational unit |
| Tree | Package | Named collection of related artifacts |
| Fruit | Instance | Specific content identified by SHA256 |
| Seed | Dependency | Required package specification |
| Harvest | Download/Fetch | Retrieve dependencies |
| Cultivate | Upload/Publish | Store new artifact |
| Graft | Alias/Tag | Alternative name for content |
| Prune | Clean/GC | Remove unused local cache |

## Future Work

The following features from the specification are planned but not yet implemented:

- [ ] CLI tool (`orchard` command)
- [ ] `orchard.ensure` file parsing
- [ ] Lock file generation (`orchard.lock`)
- [ ] Export/Import for air-gapped systems
- [ ] Consumer notification (pollination)
- [ ] Automated update propagation
- [ ] OIDC/SAML authentication
- [ ] API key management
- [ ] Package format detection
- [ ] Multipart upload for large files
- [ ] Redis caching layer

## License

Internal use only.