Add presigned URL support for direct S3 downloads (#48)

This commit is contained in:
Mondo Diaz
2025-12-15 16:06:51 -06:00
parent caa0c5af0c
commit 2df97ae94a
10 changed files with 339 additions and 43 deletions

View File

@@ -60,7 +60,8 @@ Orchard is a centralized binary artifact storage system that provides content-ad
| `GET` | `/api/v1/project/:project/packages/:package` | Get single package with metadata |
| `POST` | `/api/v1/project/:project/packages` | Create a new package |
| `POST` | `/api/v1/project/:project/:package/upload` | Upload an artifact |
| `GET` | `/api/v1/project/:project/:package/+/:ref` | Download an artifact (supports Range header) |
| `GET` | `/api/v1/project/:project/:package/+/:ref` | Download an artifact (supports Range header, mode param) |
| `GET` | `/api/v1/project/:project/:package/+/:ref/url` | Get presigned URL for direct S3 download |
| `HEAD` | `/api/v1/project/:project/:package/+/:ref` | Get artifact metadata without downloading |
| `GET` | `/api/v1/project/:project/:package/tags` | List tags (with pagination, search, sorting, artifact metadata) |
| `POST` | `/api/v1/project/:project/:package/tags` | Create a tag |
@@ -292,6 +293,12 @@ curl -H "Range: bytes=0-1023" http://localhost:8080/api/v1/project/my-project/re
# Check file info without downloading (HEAD request)
curl -I http://localhost:8080/api/v1/project/my-project/releases/+/v1.0.0
# Download with specific mode (presigned, redirect, or proxy)
curl "http://localhost:8080/api/v1/project/my-project/releases/+/v1.0.0?mode=proxy"
# Get presigned URL for direct S3 download
curl http://localhost:8080/api/v1/project/my-project/releases/+/v1.0.0/url
```
> **Note on curl flags:**
@@ -300,6 +307,33 @@ curl -I http://localhost:8080/api/v1/project/my-project/releases/+/v1.0.0
> - `-OJ` combines both: download to a file using the server-provided filename
> - `-o <filename>` saves to a specific filename you choose
#### Download Modes
Orchard supports three download modes, configurable via `ORCHARD_DOWNLOAD_MODE` or per-request with `?mode=`:
| Mode | Description | Use Case |
|------|-------------|----------|
| `presigned` (default) | Returns JSON with a presigned S3 URL | Clients that handle redirects themselves, web UIs |
| `redirect` | Returns HTTP 302 redirect to presigned S3 URL | Simple clients, browsers, wget |
| `proxy` | Streams content through the backend | When S3 isn't directly accessible to clients |
**Presigned URL Response:**
```json
{
"url": "https://minio.example.com/bucket/...",
"expires_at": "2025-01-01T01:00:00Z",
"method": "GET",
"artifact_id": "a3f5d8e...",
"size": 1048576,
"content_type": "application/gzip",
"original_name": "app-v1.0.0.tar.gz",
"checksum_sha256": "a3f5d8e...",
"checksum_md5": "d41d8cd..."
}
```
> **Note:** For presigned URLs to work, clients must be able to reach the S3 endpoint directly. In Kubernetes, this requires exposing MinIO via ingress (see Helm configuration below).
### Create a Tag
```bash
@@ -485,6 +519,8 @@ Configuration is provided via environment variables prefixed with `ORCHARD_`:
| `ORCHARD_S3_BUCKET` | S3 bucket name | `orchard-artifacts` |
| `ORCHARD_S3_ACCESS_KEY_ID` | S3 access key | - |
| `ORCHARD_S3_SECRET_ACCESS_KEY` | S3 secret key | - |
| `ORCHARD_DOWNLOAD_MODE` | Download mode: `presigned`, `redirect`, or `proxy` | `presigned` |
| `ORCHARD_PRESIGNED_URL_EXPIRY` | Presigned URL expiry in seconds | `3600` |
## Kubernetes Deployment
@@ -505,6 +541,32 @@ helm install orchard ./helm/orchard -n orchard --create-namespace
helm install orchard ./helm/orchard -f my-values.yaml
```
### Helm Configuration
Key configuration options in `values.yaml`:
```yaml
orchard:
# Download configuration
download:
mode: "presigned" # presigned, redirect, or proxy
presignedUrlExpiry: 3600
# MinIO ingress (required for presigned URL downloads)
minio:
ingress:
enabled: true
className: "nginx"
annotations:
cert-manager.io/cluster-issuer: "letsencrypt"
host: "minio.your-domain.com"
tls:
enabled: true
secretName: minio-tls
```
When `minio.ingress.enabled` is `true`, the S3 endpoint automatically uses the external URL (`https://minio.your-domain.com`), making presigned URLs accessible to external clients.
See `helm/orchard/values.yaml` for all configuration options.
## Database Schema