orchard/README.md

# Orchard

**Content-Addressable Storage System**

Orchard is a centralized binary artifact storage system that provides content-addressable storage with automatic deduplication, flexible access control, and multi-format package support.

## Tech Stack

- **Backend**: Python 3.12 + FastAPI
- **Frontend**: React 18 + TypeScript + Vite
- **Database**: PostgreSQL 16
- **Object Storage**: MinIO (S3-compatible)
- **Cache**: Redis (for future use)

## Features

### Currently Implemented

- **Content-Addressable Storage** - Artifacts are stored and referenced by their SHA256 hash, ensuring deduplication and data integrity
- **Project/Package/Artifact Hierarchy** - Organized storage structure:
  - **Project** - Top-level organizational container
  - **Package** - Named collection within a project
  - **Artifact** - Specific content instance identified by SHA256
- **Tags** - Alias system for referencing artifacts by human-readable names (e.g., `v1.0.0`, `latest`, `stable`)
- **S3-Compatible Backend** - Uses MinIO (or any S3-compatible storage) for artifact storage
- **PostgreSQL Metadata** - Relational database for metadata, access control, and audit trails
- **REST API** - Full HTTP API for all operations
- **Web UI** - React-based interface for managing artifacts
- **Docker Compose Setup** - Easy local development environment
- **Helm Chart** - Kubernetes deployment with PostgreSQL, MinIO, and Redis subcharts

### API Endpoints

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/` | Web UI |
| `GET` | `/health` | Health check |
| `GET` | `/api/v1/projects` | List all projects |
| `POST` | `/api/v1/projects` | Create a new project |
| `GET` | `/api/v1/projects/:project` | Get project details |
| `GET` | `/api/v1/project/:project/packages` | List packages in a project |
| `POST` | `/api/v1/project/:project/packages` | Create a new package |
| `POST` | `/api/v1/project/:project/:package/upload` | Upload an artifact |
| `GET` | `/api/v1/project/:project/:package/+/:ref` | Download an artifact |
| `GET` | `/api/v1/project/:project/:package/tags` | List all tags |
| `POST` | `/api/v1/project/:project/:package/tags` | Create a tag |
| `GET` | `/api/v1/project/:project/:package/consumers` | List consumers of a package |
| `GET` | `/api/v1/artifact/:id` | Get artifact metadata by hash |

### Reference Formats

When downloading artifacts, the `:ref` parameter supports multiple formats:

- `latest` - Tag name directly
- `v1.0.0` - Version tag
- `tag:stable` - Explicit tag reference
- `version:2024.1` - Version reference
- `artifact:a3f5d8e12b4c6789...` - Direct SHA256 hash reference

## Quick Start

### Prerequisites

- Docker and Docker Compose

### Running Locally

```bash
# Start all services
docker-compose up -d

# View logs
docker-compose logs -f orchard-server

# Stop services
docker-compose down
```

### Services

| Service | Port | Description |
|---------|------|-------------|
| orchard-server | 8080 | Main API server and Web UI |
| postgres | 5432 | PostgreSQL database |
| minio | 9000 | S3-compatible object storage |
| minio (console) | 9001 | MinIO web console |
| redis | 6379 | Cache (for future use) |

### Access Points

- **Web UI**: http://localhost:8080
- **API**: http://localhost:8080/api/v1
- **API Docs**: http://localhost:8080/docs
- **MinIO Console**: http://localhost:9001 (user: `minioadmin`, pass: `minioadmin`)

## Development

### Backend (FastAPI)

```bash
cd backend
pip install -r requirements.txt
uvicorn app.main:app --reload --port 8080
```

### Frontend (React)

```bash
cd frontend
npm install
npm run dev
```

The frontend dev server proxies API requests to `localhost:8080`.

## Usage Examples

### Create a Project

```bash
curl -X POST http://localhost:8080/api/v1/projects \
  -H "Content-Type: application/json" \
  -d '{"name": "my-project", "description": "My project artifacts", "is_public": true}'
```

### Create a Package

```bash
curl -X POST http://localhost:8080/api/v1/project/my-project/packages \
  -H "Content-Type: application/json" \
  -d '{"name": "releases", "description": "Release builds"}'
```

### Upload an Artifact

```bash
curl -X POST http://localhost:8080/api/v1/project/my-project/releases/upload \
  -F "file=@./build/app-v1.0.0.tar.gz" \
  -F "tag=v1.0.0"
```

Response:
```json
{
  "artifact_id": "a3f5d8e12b4c67890abcdef1234567890abcdef1234567890abcdef12345678",
  "size": 1048576,
  "project": "my-project",
  "package": "releases",
  "tag": "v1.0.0"
}
```

### Download an Artifact

```bash
# By tag
curl -O http://localhost:8080/api/v1/project/my-project/releases/+/v1.0.0

# By artifact ID
curl -O http://localhost:8080/api/v1/project/my-project/releases/+/artifact:a3f5d8e12b4c6789...

# Using the short URL pattern
curl -O http://localhost:8080/project/my-project/releases/+/latest
```

### Create a Tag

```bash
curl -X POST http://localhost:8080/api/v1/project/my-project/releases/tags \
  -H "Content-Type: application/json" \
  -d '{"name": "stable", "artifact_id": "a3f5d8e12b4c6789..."}'
```

### Get Artifact by ID

```bash
curl http://localhost:8080/api/v1/artifact/a3f5d8e12b4c67890abcdef1234567890abcdef1234567890abcdef12345678
```

## Project Structure

```
orchard/
├── backend/
│   ├── app/
│   │   ├── __init__.py
│   │   ├── config.py           # Pydantic settings
│   │   ├── database.py         # SQLAlchemy setup
│   │   ├── main.py             # FastAPI application
│   │   ├── models.py           # SQLAlchemy models
│   │   ├── routes.py           # API endpoints
│   │   ├── schemas.py          # Pydantic schemas
│   │   └── storage.py          # S3 storage layer
│   └── requirements.txt
├── frontend/
│   ├── src/
│   │   ├── components/         # React components
│   │   ├── pages/              # Page components
│   │   ├── api.ts              # API client
│   │   ├── types.ts            # TypeScript types
│   │   ├── App.tsx
│   │   └── main.tsx
│   ├── index.html
│   ├── package.json
│   ├── tsconfig.json
│   └── vite.config.ts
├── helm/
│   └── orchard/                # Helm chart
├── Dockerfile                  # Multi-stage build (Node + Python)
├── docker-compose.yml          # Local development stack
└── .gitlab-ci.yml              # CI/CD pipeline
```

## Configuration

Configuration is provided via environment variables prefixed with `ORCHARD_`:

| Environment Variable | Description | Default |
|---------------------|-------------|---------|
| `ORCHARD_SERVER_HOST` | Server bind address | `0.0.0.0` |
| `ORCHARD_SERVER_PORT` | Server port | `8080` |
| `ORCHARD_DATABASE_HOST` | PostgreSQL host | `localhost` |
| `ORCHARD_DATABASE_PORT` | PostgreSQL port | `5432` |
| `ORCHARD_DATABASE_USER` | PostgreSQL user | `orchard` |
| `ORCHARD_DATABASE_PASSWORD` | PostgreSQL password | - |
| `ORCHARD_DATABASE_DBNAME` | PostgreSQL database | `orchard` |
| `ORCHARD_S3_ENDPOINT` | S3 endpoint URL | - |
| `ORCHARD_S3_REGION` | S3 region | `us-east-1` |
| `ORCHARD_S3_BUCKET` | S3 bucket name | `orchard-artifacts` |
| `ORCHARD_S3_ACCESS_KEY_ID` | S3 access key | - |
| `ORCHARD_S3_SECRET_ACCESS_KEY` | S3 secret key | - |

## Kubernetes Deployment

### Using Helm

```bash
# Add Bitnami repo for dependencies
helm repo add bitnami https://charts.bitnami.com/bitnami

# Update dependencies
cd helm/orchard
helm dependency update

# Install
helm install orchard ./helm/orchard -n orchard --create-namespace

# Install with custom values
helm install orchard ./helm/orchard -f my-values.yaml
```

See `helm/orchard/values.yaml` for all configuration options.

## Database Schema

### Core Tables

- **projects** - Top-level organizational containers
- **packages** - Collections within projects
- **artifacts** - Content-addressable artifacts (SHA256)
- **tags** - Aliases pointing to artifacts
- **tag_history** - Audit trail for tag changes
- **uploads** - Upload event records
- **consumers** - Dependency tracking
- **access_permissions** - Project-level access control
- **api_keys** - Programmatic access tokens
- **audit_logs** - Immutable operation logs

## Future Work

The following features are planned but not yet implemented:

- [ ] CLI tool (`orchard` command)
- [ ] Dependency file parsing
- [ ] Lock file generation
- [ ] Export/Import for air-gapped systems
- [ ] Consumer notification
- [ ] Automated update propagation
- [ ] OIDC/SAML authentication
- [ ] API key management
- [ ] Package format detection
- [ ] Multipart upload for large files
- [ ] Redis caching layer

## License

Internal use only.