bitforge/cf-uploader
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Dynamic size

Browse Source
This commit is contained in:
pratik
2025-10-21 18:48:35 -05:00
parent 6e3684b580
commit b81ab51100
3 changed files with 40 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files

32
CHUNKED_UPLOAD_GUIDE.md
View File

@@ -101,7 +101,13 @@ Same as the traditional `/api/cf/deploy` endpoint.
## Client Implementation Example (JavaScript)
```javascript
const CHUNK_SIZE = 5 * 1024 * 1024; // 5MB
// You can use ANY chunk size - server supports variable chunk sizes!
// Recommended: 1-2MB for Tanzu with memory constraints
const CHUNK_SIZE = 1 * 1024 * 1024; // 1MB
// Other options:
// const CHUNK_SIZE = 512 * 1024; // 512KB (very safe)
// const CHUNK_SIZE = 2 * 1024 * 1024; // 2MB (balanced)
// const CHUNK_SIZE = 5 * 1024 * 1024; // 5MB (if you have memory)
async function deployWithChunks(jarFile, manifestFile, deploymentConfig) {
const apiBase = 'https://your-app.example.com/api/cf';
@@ -191,7 +197,8 @@ deployWithChunks(jarInput.files[0], manifestInput.files[0], config)
import requests
import os
CHUNK_SIZE = 5 * 1024 * 1024 # 5MB
# You can use ANY chunk size!
CHUNK_SIZE = 1 * 1024 * 1024 # 1MB (recommended for Tanzu)
def deploy_with_chunks(api_base, jar_path, manifest_path, deployment_config):
# Step 1: Initialize upload session
@@ -316,13 +323,13 @@ server {
```properties
# Chunked Upload Configuration
cf.upload.chunk.size=5242880
cf.upload.session.timeout-minutes=30
```
- **cf.upload.chunk.size**: Size of each chunk in bytes (default: 5MB)
- **cf.upload.session.timeout-minutes**: How long inactive sessions are kept (default: 30 minutes)
**Note:** There is NO server-side chunk size configuration. The server accepts ANY chunk size from the client. Chunks are appended sequentially as they arrive.
## Session Management
- Upload sessions expire after 30 minutes of inactivity (configurable)
@@ -356,17 +363,22 @@ The traditional `/api/cf/deploy` endpoint remains available and functional. You
## Performance Considerations
- **Chunk size**: 5MB is a good balance between request count and size
- Smaller chunks = more requests but safer for proxies
- Larger chunks = fewer requests but may hit proxy limits
- **Chunk size**: Client controls this completely
- **Smaller chunks (512KB-1MB)**: More requests, but safer for memory-constrained servers and strict proxies
- **Larger chunks (5-10MB)**: Fewer requests, faster uploads, but needs more memory
- **Recommended for Tanzu**: 1MB (good balance for low-memory environments)
- **Any size works**: Server accepts variable chunk sizes
- **Parallel uploads**: Current implementation is sequential
- Files are uploaded one chunk at a time
- Chunks must be uploaded in order (0, 1, 2, ...)
- **Sequential upload requirement**: **CRITICAL**
- Chunks **MUST** be uploaded in order: 0, 1, 2, 3...
- Server validates and enforces sequential order
- Out-of-order chunks will be rejected
- This is necessary because chunks are appended sequentially to the file
- **Network reliability**: Chunked uploads are more resilient
- Failed chunks can be retried individually
- No need to re-upload the entire file on failure
- Just retry the specific failed chunk index
## Monitoring