Update documentation for upload/download API enhancements

- Add resumable upload endpoints to API table
- Add HEAD request endpoint documentation
- Document format-specific metadata extraction feature
- Add resumable upload usage examples
- Add range request and HEAD request examples
- Update project structure with metadata.py
- Update Future Work section (remove completed items)
This commit is contained in:
Mondo Diaz
2025-12-11 17:09:33 -06:00
parent 6eb2f9db7b
commit ec0789c92f

View File

@@ -28,6 +28,16 @@ Orchard is a centralized binary artifact storage system that provides content-ad
- **Web UI** - React-based interface for managing artifacts
- **Docker Compose Setup** - Easy local development environment
- **Helm Chart** - Kubernetes deployment with PostgreSQL, MinIO, and Redis subcharts
- **Multipart Upload** - Automatic multipart upload for files larger than 100MB
- **Resumable Uploads** - API for resumable uploads with part-by-part upload support
- **Range Requests** - HTTP range request support for partial downloads
- **Format-Specific Metadata** - Automatic extraction of metadata from package formats:
- `.deb` - Debian packages (name, version, architecture, maintainer)
- `.rpm` - RPM packages (name, version, release, architecture)
- `.tar.gz/.tgz` - Tarballs (name, version from filename)
- `.whl` - Python wheels (name, version, author)
- `.jar` - Java JARs (manifest info, Maven coordinates)
- `.zip` - ZIP files (file count, uncompressed size)
### API Endpoints
@@ -41,12 +51,25 @@ Orchard is a centralized binary artifact storage system that provides content-ad
| `GET` | `/api/v1/project/:project/packages` | List packages in a project |
| `POST` | `/api/v1/project/:project/packages` | Create a new package |
| `POST` | `/api/v1/project/:project/:package/upload` | Upload an artifact |
| `GET` | `/api/v1/project/:project/:package/+/:ref` | Download an artifact |
| `GET` | `/api/v1/project/:project/:package/+/:ref` | Download an artifact (supports Range header) |
| `HEAD` | `/api/v1/project/:project/:package/+/:ref` | Get artifact metadata without downloading |
| `GET` | `/api/v1/project/:project/:package/tags` | List all tags |
| `POST` | `/api/v1/project/:project/:package/tags` | Create a tag |
| `GET` | `/api/v1/project/:project/:package/consumers` | List consumers of a package |
| `GET` | `/api/v1/artifact/:id` | Get artifact metadata by hash |
#### Resumable Upload Endpoints
For large files, use the resumable upload API:
| Method | Endpoint | Description |
|--------|----------|-------------|
| `POST` | `/api/v1/project/:project/:package/upload/init` | Initialize resumable upload |
| `PUT` | `/api/v1/project/:project/:package/upload/:upload_id/part/:part_number` | Upload a part |
| `POST` | `/api/v1/project/:project/:package/upload/:upload_id/complete` | Complete upload |
| `DELETE` | `/api/v1/project/:project/:package/upload/:upload_id` | Abort upload |
| `GET` | `/api/v1/project/:project/:package/upload/:upload_id/status` | Get upload status |
### Reference Formats
When downloading artifacts, the `:ref` parameter supports multiple formats:
@@ -146,10 +169,43 @@ Response:
"size": 1048576,
"project": "my-project",
"package": "releases",
"tag": "v1.0.0"
"tag": "v1.0.0",
"format_metadata": {
"format": "tarball",
"package_name": "app",
"version": "1.0.0"
},
"deduplicated": false
}
```
### Resumable Upload (for large files)
For files larger than 100MB, use the resumable upload API:
```bash
# 1. Initialize upload (client must compute SHA256 hash first)
curl -X POST http://localhost:8080/api/v1/project/my-project/releases/upload/init \
-H "Content-Type: application/json" \
-d '{
"expected_hash": "a3f5d8e12b4c67890abcdef1234567890abcdef1234567890abcdef12345678",
"filename": "large-file.tar.gz",
"size": 524288000,
"tag": "v2.0.0"
}'
# Response: {"upload_id": "abc123", "already_exists": false, "chunk_size": 10485760}
# 2. Upload parts (10MB chunks recommended)
curl -X PUT http://localhost:8080/api/v1/project/my-project/releases/upload/abc123/part/1 \
--data-binary @chunk1.bin
# 3. Complete the upload
curl -X POST http://localhost:8080/api/v1/project/my-project/releases/upload/abc123/complete \
-H "Content-Type: application/json" \
-d '{"tag": "v2.0.0"}'
```
### Download an Artifact
```bash
@@ -161,6 +217,12 @@ curl -O http://localhost:8080/api/v1/project/my-project/releases/+/artifact:a3f5
# Using the short URL pattern
curl -O http://localhost:8080/project/my-project/releases/+/latest
# Partial download (range request)
curl -H "Range: bytes=0-1023" http://localhost:8080/api/v1/project/my-project/releases/+/v1.0.0
# Check file info without downloading (HEAD request)
curl -I http://localhost:8080/api/v1/project/my-project/releases/+/v1.0.0
```
### Create a Tag
@@ -185,12 +247,13 @@ orchard/
│ ├── app/
│ │ ├── __init__.py
│ │ ├── config.py # Pydantic settings
│ │ ├── database.py # SQLAlchemy setup
│ │ ├── database.py # SQLAlchemy setup and migrations
│ │ ├── main.py # FastAPI application
│ │ ├── metadata.py # Format-specific metadata extraction
│ │ ├── models.py # SQLAlchemy models
│ │ ├── routes.py # API endpoints
│ │ ├── schemas.py # Pydantic schemas
│ │ └── storage.py # S3 storage layer
│ │ └── storage.py # S3 storage layer with multipart support
│ └── requirements.txt
├── frontend/
│ ├── src/
@@ -278,9 +341,8 @@ The following features are planned but not yet implemented:
- [ ] Automated update propagation
- [ ] OIDC/SAML authentication
- [ ] API key management
- [ ] Package format detection
- [ ] Multipart upload for large files
- [ ] Redis caching layer
- [ ] Garbage collection for orphaned artifacts
## License