Additional files

2025-10-22 09:18:11 -05:00
parent 34567cd813
commit 72708bbf91
6 changed files with 695 additions and 0 deletions
--- a/TIMEOUT_SOLUTION.md
+++ b/TIMEOUT_SOLUTION.md
@@ -0,0 +1,281 @@
+# Nginx Timeout Solution
+
+## The Problem
+
+### Chunking Solves Upload Timeouts ✅
+- Each chunk upload completes in seconds
+- Well under nginx's 30-second timeout
+- **No problem here!**
+
+### But Deployment Still Times Out ❌
+The `/upload/finalize` endpoint can take 3-5+ minutes because:
+1. CF login
+2. CF push (staging, building, deploying)
+3. CF logout
+
+This **WILL** hit nginx's 30-second timeout!
+
+## The Solution: Async Deployment
+
+Instead of waiting for deployment to complete, we return immediately and let the client poll for status.
+
+### Flow Comparison
+
+**Before (Times Out):**
+```
+Client → finalize → [waits 5 minutes] → ⏱️ NGINX TIMEOUT after 30s
+```
+
+**After (Works):**
+```
+Client → finalize → ✅ Returns immediately (202 Accepted)
+Client → poll status every 5s → IN_PROGRESS
+Client → poll status → IN_PROGRESS
+Client → poll status → COMPLETED ✅
+```
+
+## Updated API
+
+### 1. Initialize Upload (unchanged)
+```bash
+POST /api/cf/upload/init
+```
+
+### 2. Upload Chunks (unchanged)
+```bash
+POST /api/cf/upload/chunk
+```
+
+### 3. Finalize Upload (NEW: async by default)
+```bash
+POST /api/cf/upload/finalize?uploadSessionId={sessionId}&async=true
+```
+
+**Response (202 Accepted):**
+```json
+{
+  "uploadSessionId": "550e8400-e29b-41d4-a716-446655440000",
+  "status": "IN_PROGRESS",
+  "message": "Deployment started. Use /deployment/status endpoint to check progress.",
+  "progress": 0
+}
+```
+
+### 4. Poll Deployment Status (NEW)
+```bash
+GET /api/cf/deployment/status/{uploadSessionId}
+```
+
+**Response while deploying:**
+```json
+{
+  "uploadSessionId": "550e8400-e29b-41d4-a716-446655440000",
+  "status": "IN_PROGRESS",
+  "message": "Logging into Cloud Foundry...",
+  "progress": 10
+}
+```
+
+**Response when complete:**
+```json
+{
+  "uploadSessionId": "550e8400-e29b-41d4-a716-446655440000",
+  "status": "COMPLETED",
+  "message": "Deployment completed successfully",
+  "output": "[full CF CLI output]",
+  "progress": 100
+}
+```
+
+**Response if failed:**
+```json
+{
+  "uploadSessionId": "550e8400-e29b-41d4-a716-446655440000",
+  "status": "FAILED",
+  "message": "Deployment failed: ...",
+  "error": "[error details]",
+  "progress": 0
+}
+```
+
+## Updated Bash Script
+
+```bash
+#!/bin/bash
+
+API_BASE="http://your-app.example.com/api/cf"
+JAR_FILE="hf.jar"
+MANIFEST_FILE="manifest.yml"
+CHUNK_SIZE=1048576  # 1MB
+
+CF_CONFIG='{
+  "apiEndpoint": "https://api.cf.example.com",
+  "username": "your-username",
+  "password": "your-password",
+  "organization": "your-org",
+  "space": "your-space",
+  "appName": "your-app",
+  "skipSslValidation": false
+}'
+
+echo "=== Step 1: Initialize Upload Session ==="
+INIT_RESPONSE=$(curl -s -X POST "$API_BASE/upload/init" \
+  -H "Content-Type: application/json" \
+  -d "$CF_CONFIG")
+
+SESSION_ID=$(echo $INIT_RESPONSE | grep -o '"uploadSessionId":"[^"]*' | cut -d'"' -f4)
+echo "Session created: $SESSION_ID"
+
+# Function to upload file in chunks
+upload_file_in_chunks() {
+  local file_path=$1
+  local file_type=$2
+  local file_name=$(basename "$file_path")
+  local file_size=$(stat -f%z "$file_path" 2>/dev/null || stat -c%s "$file_path")
+  local total_chunks=$(( ($file_size + $CHUNK_SIZE - 1) / $CHUNK_SIZE ))
+
+  echo ""
+  echo "=== Uploading $file_type: $file_name ($total_chunks chunks) ==="
+
+  local temp_dir=$(mktemp -d)
+  split -b $CHUNK_SIZE "$file_path" "$temp_dir/chunk_"
+
+  local chunk_index=0
+  for chunk_file in "$temp_dir"/chunk_*; do
+    printf "Uploading chunk %3d/%3d... " "$((chunk_index + 1))" "$total_chunks"
+
+    RESPONSE=$(curl -s -X POST "$API_BASE/upload/chunk" \
+      -F "uploadSessionId=$SESSION_ID" \
+      -F "fileType=$file_type" \
+      -F "chunkIndex=$chunk_index" \
+      -F "totalChunks=$total_chunks" \
+      -F "fileName=$file_name" \
+      -F "chunk=@$chunk_file")
+
+    SUCCESS=$(echo $RESPONSE | grep -o '"success":[^,]*' | cut -d':' -f2)
+
+    if [ "$SUCCESS" != "true" ]; then
+      echo "FAILED"
+      echo "$RESPONSE"
+      rm -rf "$temp_dir"
+      exit 1
+    fi
+
+    echo "OK"
+    chunk_index=$((chunk_index + 1))
+  done
+
+  rm -rf "$temp_dir"
+  echo "$file_type upload completed"
+}
+
+# Step 2: Upload JAR file
+upload_file_in_chunks "$JAR_FILE" "jarFile"
+
+# Step 3: Upload manifest file
+upload_file_in_chunks "$MANIFEST_FILE" "manifest"
+
+# Step 4: Start async deployment
+echo ""
+echo "=== Step 4: Starting deployment (async) ==="
+FINALIZE_RESPONSE=$(curl -s -X POST "$API_BASE/upload/finalize?uploadSessionId=$SESSION_ID&async=true")
+
+STATUS=$(echo $FINALIZE_RESPONSE | grep -o '"status":"[^"]*' | cut -d'"' -f4)
+
+if [ "$STATUS" != "IN_PROGRESS" ]; then
+  echo "Failed to start deployment:"
+  echo "$FINALIZE_RESPONSE"
+  exit 1
+fi
+
+echo "Deployment started. Polling for status..."
+
+# Step 5: Poll deployment status
+POLL_INTERVAL=5  # seconds
+MAX_WAIT=600     # 10 minutes max
+
+elapsed=0
+while [ $elapsed -lt $MAX_WAIT ]; do
+  sleep $POLL_INTERVAL
+  elapsed=$((elapsed + POLL_INTERVAL))
+
+  STATUS_RESPONSE=$(curl -s "$API_BASE/deployment/status/$SESSION_ID")
+  CURRENT_STATUS=$(echo $STATUS_RESPONSE | grep -o '"status":"[^"]*' | cut -d'"' -f4)
+  MESSAGE=$(echo $STATUS_RESPONSE | grep -o '"message":"[^"]*' | cut -d'"' -f4)
+  PROGRESS=$(echo $STATUS_RESPONSE | grep -o '"progress":[0-9]*' | cut -d':' -f2)
+
+  printf "\r[%3ds] Status: %-15s Progress: %3s%% - %s" \
+    "$elapsed" "$CURRENT_STATUS" "${PROGRESS:-0}" "$MESSAGE"
+
+  if [ "$CURRENT_STATUS" = "COMPLETED" ]; then
+    echo ""
+    echo ""
+    echo "=== Deployment successful! ==="
+    echo "$STATUS_RESPONSE" | jq '.' 2>/dev/null || echo "$STATUS_RESPONSE"
+    exit 0
+  elif [ "$CURRENT_STATUS" = "FAILED" ]; then
+    echo ""
+    echo ""
+    echo "=== Deployment failed ==="
+    echo "$STATUS_RESPONSE" | jq '.' 2>/dev/null || echo "$STATUS_RESPONSE"
+    exit 1
+  fi
+done
+
+echo ""
+echo "=== Deployment timeout after ${MAX_WAIT}s ==="
+echo "Check status manually: curl $API_BASE/deployment/status/$SESSION_ID"
+exit 1
+```
+
+## Status Values
+
+| Status | Description |
+|--------|-------------|
+| `PENDING` | Upload session created but deployment not started |
+| `IN_PROGRESS` | Deployment is currently running |
+| `COMPLETED` | Deployment finished successfully |
+| `FAILED` | Deployment failed with errors |
+
+## Nginx Configuration
+
+With async deployment, nginx timeout is **not an issue**:
+
+```nginx
+server {
+    listen 80;
+    server_name your-app.example.com;
+
+    # Each chunk upload completes quickly
+    client_max_body_size 10m;
+
+    # Standard timeouts work fine now
+    proxy_read_timeout 60s;    # Chunks complete in <5s
+    proxy_connect_timeout 10s;
+    proxy_send_timeout 60s;
+
+    location /api/cf/ {
+        proxy_pass http://cf-deployer-backend:8080;
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+    }
+}
+```
+
+## For Backwards Compatibility
+
+If you want synchronous deployment (will timeout on nginx!):
+
+```bash
+# Synchronous (old way - may timeout)
+curl -X POST "$API_BASE/upload/finalize?uploadSessionId=$SESSION_ID&async=false"
+```
+
+**Default is async=true** to avoid timeout issues.
+
+## Summary
+
+✅ **Chunk uploads**: Complete in seconds, no timeout
+✅ **Finalize endpoint**: Returns immediately (async), no timeout
+✅ **Status polling**: Each poll completes in milliseconds, no timeout
+✅ **Total solution**: Works with standard 30-second nginx timeout!