Implement backend upload/download API enhancements

README.md

@@ -28,6 +28,16 @@ Orchard is a centralized binary artifact storage system that provides content-ad
 - **Web UI** - React-based interface for managing artifacts
 - **Docker Compose Setup** - Easy local development environment
 - **Helm Chart** - Kubernetes deployment with PostgreSQL, MinIO, and Redis subcharts
+- **Multipart Upload** - Automatic multipart upload for files larger than 100MB
+- **Resumable Uploads** - API for resumable uploads with part-by-part upload support
+- **Range Requests** - HTTP range request support for partial downloads
+- **Format-Specific Metadata** - Automatic extraction of metadata from package formats:
+  - `.deb` - Debian packages (name, version, architecture, maintainer)
+  - `.rpm` - RPM packages (name, version, release, architecture)
+  - `.tar.gz/.tgz` - Tarballs (name, version from filename)
+  - `.whl` - Python wheels (name, version, author)
+  - `.jar` - Java JARs (manifest info, Maven coordinates)
+  - `.zip` - ZIP files (file count, uncompressed size)
 
 ### API Endpoints
 
@@ -41,12 +51,25 @@ Orchard is a centralized binary artifact storage system that provides content-ad
 | `GET` | `/api/v1/project/:project/packages` | List packages in a project |
 | `POST` | `/api/v1/project/:project/packages` | Create a new package |
 | `POST` | `/api/v1/project/:project/:package/upload` | Upload an artifact |
-| `GET` | `/api/v1/project/:project/:package/+/:ref` | Download an artifact |
+| `GET` | `/api/v1/project/:project/:package/+/:ref` | Download an artifact (supports Range header) |
+| `HEAD` | `/api/v1/project/:project/:package/+/:ref` | Get artifact metadata without downloading |
 | `GET` | `/api/v1/project/:project/:package/tags` | List all tags |
 | `POST` | `/api/v1/project/:project/:package/tags` | Create a tag |
 | `GET` | `/api/v1/project/:project/:package/consumers` | List consumers of a package |
 | `GET` | `/api/v1/artifact/:id` | Get artifact metadata by hash |
 
+#### Resumable Upload Endpoints
+
+For large files, use the resumable upload API:
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| `POST` | `/api/v1/project/:project/:package/upload/init` | Initialize resumable upload |
+| `PUT` | `/api/v1/project/:project/:package/upload/:upload_id/part/:part_number` | Upload a part |
+| `POST` | `/api/v1/project/:project/:package/upload/:upload_id/complete` | Complete upload |
+| `DELETE` | `/api/v1/project/:project/:package/upload/:upload_id` | Abort upload |
+| `GET` | `/api/v1/project/:project/:package/upload/:upload_id/status` | Get upload status |
+
 ### Reference Formats
 
 When downloading artifacts, the `:ref` parameter supports multiple formats:
@@ -146,10 +169,43 @@ Response:
   "size": 1048576,
   "project": "my-project",
   "package": "releases",
-  "tag": "v1.0.0"
+  "tag": "v1.0.0",
+  "format_metadata": {
+    "format": "tarball",
+    "package_name": "app",
+    "version": "1.0.0"
+  },
+  "deduplicated": false
 }
 ```
 
+### Resumable Upload (for large files)
+
+For files larger than 100MB, use the resumable upload API:
+
+```bash
+# 1. Initialize upload (client must compute SHA256 hash first)
+curl -X POST http://localhost:8080/api/v1/project/my-project/releases/upload/init \
+  -H "Content-Type: application/json" \
+  -d '{
+    "expected_hash": "a3f5d8e12b4c67890abcdef1234567890abcdef1234567890abcdef12345678",
+    "filename": "large-file.tar.gz",
+    "size": 524288000,
+    "tag": "v2.0.0"
+  }'
+
+# Response: {"upload_id": "abc123", "already_exists": false, "chunk_size": 10485760}
+
+# 2. Upload parts (10MB chunks recommended)
+curl -X PUT http://localhost:8080/api/v1/project/my-project/releases/upload/abc123/part/1 \
+  --data-binary @chunk1.bin
+
+# 3. Complete the upload
+curl -X POST http://localhost:8080/api/v1/project/my-project/releases/upload/abc123/complete \
+  -H "Content-Type: application/json" \
+  -d '{"tag": "v2.0.0"}'
+```
+
 ### Download an Artifact
 
 ```bash
@@ -161,6 +217,12 @@ curl -O http://localhost:8080/api/v1/project/my-project/releases/+/artifact:a3f5
 
 # Using the short URL pattern
 curl -O http://localhost:8080/project/my-project/releases/+/latest
+
+# Partial download (range request)
+curl -H "Range: bytes=0-1023" http://localhost:8080/api/v1/project/my-project/releases/+/v1.0.0
+
+# Check file info without downloading (HEAD request)
+curl -I http://localhost:8080/api/v1/project/my-project/releases/+/v1.0.0
 ```
 
 ### Create a Tag
@@ -185,12 +247,13 @@ orchard/
 │   ├── app/
 │   │   ├── __init__.py
 │   │   ├── config.py           # Pydantic settings
-│   │   ├── database.py         # SQLAlchemy setup
+│   │   ├── database.py         # SQLAlchemy setup and migrations
 │   │   ├── main.py             # FastAPI application
+│   │   ├── metadata.py         # Format-specific metadata extraction
 │   │   ├── models.py           # SQLAlchemy models
 │   │   ├── routes.py           # API endpoints
 │   │   ├── schemas.py          # Pydantic schemas
-│   │   └── storage.py          # S3 storage layer
+│   │   └── storage.py          # S3 storage layer with multipart support
 │   └── requirements.txt
 ├── frontend/
 │   ├── src/
@@ -278,9 +341,8 @@ The following features are planned but not yet implemented:
 - [ ] Automated update propagation
 - [ ] OIDC/SAML authentication
 - [ ] API key management
-- [ ] Package format detection
-- [ ] Multipart upload for large files
 - [ ] Redis caching layer
+- [ ] Garbage collection for orphaned artifacts
 
 ## License