cf-uploader/TIMEOUT_SOLUTION.md

Nginx Timeout Solution

The Problem

Chunking Solves Upload Timeouts ✅

• Each chunk upload completes in seconds
• Well under nginx's 30-second timeout
• No problem here!

But Deployment Still Times Out ❌

The /upload/finalize endpoint can take 3-5+ minutes because:

1. CF login
2. CF push (staging, building, deploying)
3. CF logout

This WILL hit nginx's 30-second timeout!

The Solution: Async Deployment

Instead of waiting for deployment to complete, we return immediately and let the client poll for status.

Flow Comparison

Before (Times Out):

Client → finalize → [waits 5 minutes] → ⏱️ NGINX TIMEOUT after 30s

After (Works):

Client → finalize → ✅ Returns immediately (202 Accepted)
Client → poll status every 5s → IN_PROGRESS
Client → poll status → IN_PROGRESS
Client → poll status → COMPLETED ✅

Updated API

1. Initialize Upload (unchanged)

POST /api/cf/upload/init

2. Upload Chunks (unchanged)

POST /api/cf/upload/chunk

3. Finalize Upload (NEW: async by default)

POST /api/cf/upload/finalize?uploadSessionId={sessionId}&async=true

Response (202 Accepted):

{
  "uploadSessionId": "550e8400-e29b-41d4-a716-446655440000",
  "status": "IN_PROGRESS",
  "message": "Deployment started. Use /deployment/status endpoint to check progress.",
  "progress": 0
}

4. Poll Deployment Status (NEW)

GET /api/cf/deployment/status/{uploadSessionId}

Response while deploying:

{
  "uploadSessionId": "550e8400-e29b-41d4-a716-446655440000",
  "status": "IN_PROGRESS",
  "message": "Logging into Cloud Foundry...",
  "progress": 10
}

Response when complete:

{
  "uploadSessionId": "550e8400-e29b-41d4-a716-446655440000",
  "status": "COMPLETED",
  "message": "Deployment completed successfully",
  "output": "[full CF CLI output]",
  "progress": 100
}

Response if failed:

{
  "uploadSessionId": "550e8400-e29b-41d4-a716-446655440000",
  "status": "FAILED",
  "message": "Deployment failed: ...",
  "error": "[error details]",
  "progress": 0
}

Updated Bash Script

#!/bin/bash

API_BASE="http://your-app.example.com/api/cf"
JAR_FILE="hf.jar"
MANIFEST_FILE="manifest.yml"
CHUNK_SIZE=1048576  # 1MB

CF_CONFIG='{
  "apiEndpoint": "https://api.cf.example.com",
  "username": "your-username",
  "password": "your-password",
  "organization": "your-org",
  "space": "your-space",
  "appName": "your-app",
  "skipSslValidation": false
}'

echo "=== Step 1: Initialize Upload Session ==="
INIT_RESPONSE=$(curl -s -X POST "$API_BASE/upload/init" \
  -H "Content-Type: application/json" \
  -d "$CF_CONFIG")

SESSION_ID=$(echo $INIT_RESPONSE | grep -o '"uploadSessionId":"[^"]*' | cut -d'"' -f4)
echo "Session created: $SESSION_ID"

# Function to upload file in chunks
upload_file_in_chunks() {
  local file_path=$1
  local file_type=$2
  local file_name=$(basename "$file_path")
  local file_size=$(stat -f%z "$file_path" 2>/dev/null || stat -c%s "$file_path")
  local total_chunks=$(( ($file_size + $CHUNK_SIZE - 1) / $CHUNK_SIZE ))

  echo ""
  echo "=== Uploading $file_type: $file_name ($total_chunks chunks) ==="

  local temp_dir=$(mktemp -d)
  split -b $CHUNK_SIZE "$file_path" "$temp_dir/chunk_"

  local chunk_index=0
  for chunk_file in "$temp_dir"/chunk_*; do
    printf "Uploading chunk %3d/%3d... " "$((chunk_index + 1))" "$total_chunks"

    RESPONSE=$(curl -s -X POST "$API_BASE/upload/chunk" \
      -F "uploadSessionId=$SESSION_ID" \
      -F "fileType=$file_type" \
      -F "chunkIndex=$chunk_index" \
      -F "totalChunks=$total_chunks" \
      -F "fileName=$file_name" \
      -F "chunk=@$chunk_file")

    SUCCESS=$(echo $RESPONSE | grep -o '"success":[^,]*' | cut -d':' -f2)

    if [ "$SUCCESS" != "true" ]; then
      echo "FAILED"
      echo "$RESPONSE"
      rm -rf "$temp_dir"
      exit 1
    fi

    echo "OK"
    chunk_index=$((chunk_index + 1))
  done

  rm -rf "$temp_dir"
  echo "$file_type upload completed"
}

# Step 2: Upload JAR file
upload_file_in_chunks "$JAR_FILE" "jarFile"

# Step 3: Upload manifest file
upload_file_in_chunks "$MANIFEST_FILE" "manifest"

# Step 4: Start async deployment
echo ""
echo "=== Step 4: Starting deployment (async) ==="
FINALIZE_RESPONSE=$(curl -s -X POST "$API_BASE/upload/finalize?uploadSessionId=$SESSION_ID&async=true")

STATUS=$(echo $FINALIZE_RESPONSE | grep -o '"status":"[^"]*' | cut -d'"' -f4)

if [ "$STATUS" != "IN_PROGRESS" ]; then
  echo "Failed to start deployment:"
  echo "$FINALIZE_RESPONSE"
  exit 1
fi

echo "Deployment started. Polling for status..."

# Step 5: Poll deployment status
POLL_INTERVAL=5  # seconds
MAX_WAIT=600     # 10 minutes max

elapsed=0
while [ $elapsed -lt $MAX_WAIT ]; do
  sleep $POLL_INTERVAL
  elapsed=$((elapsed + POLL_INTERVAL))

  STATUS_RESPONSE=$(curl -s "$API_BASE/deployment/status/$SESSION_ID")
  CURRENT_STATUS=$(echo $STATUS_RESPONSE | grep -o '"status":"[^"]*' | cut -d'"' -f4)
  MESSAGE=$(echo $STATUS_RESPONSE | grep -o '"message":"[^"]*' | cut -d'"' -f4)
  PROGRESS=$(echo $STATUS_RESPONSE | grep -o '"progress":[0-9]*' | cut -d':' -f2)

  printf "\r[%3ds] Status: %-15s Progress: %3s%% - %s" \
    "$elapsed" "$CURRENT_STATUS" "${PROGRESS:-0}" "$MESSAGE"

  if [ "$CURRENT_STATUS" = "COMPLETED" ]; then
    echo ""
    echo ""
    echo "=== Deployment successful! ==="
    echo "$STATUS_RESPONSE" | jq '.' 2>/dev/null || echo "$STATUS_RESPONSE"
    exit 0
  elif [ "$CURRENT_STATUS" = "FAILED" ]; then
    echo ""
    echo ""
    echo "=== Deployment failed ==="
    echo "$STATUS_RESPONSE" | jq '.' 2>/dev/null || echo "$STATUS_RESPONSE"
    exit 1
  fi
done

echo ""
echo "=== Deployment timeout after ${MAX_WAIT}s ==="
echo "Check status manually: curl $API_BASE/deployment/status/$SESSION_ID"
exit 1

Status Values

Status	Description
PENDING	Upload session created but deployment not started
IN_PROGRESS	Deployment is currently running
COMPLETED	Deployment finished successfully
FAILED	Deployment failed with errors

Nginx Configuration

With async deployment, nginx timeout is not an issue:

server {
    listen 80;
    server_name your-app.example.com;

    # Each chunk upload completes quickly
    client_max_body_size 10m;

    # Standard timeouts work fine now
    proxy_read_timeout 60s;    # Chunks complete in <5s
    proxy_connect_timeout 10s;
    proxy_send_timeout 60s;

    location /api/cf/ {
        proxy_pass http://cf-deployer-backend:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

For Backwards Compatibility

If you want synchronous deployment (will timeout on nginx!):

# Synchronous (old way - may timeout)
curl -X POST "$API_BASE/upload/finalize?uploadSessionId=$SESSION_ID&async=false"

Default is async=true to avoid timeout issues.

Summary

✅ Chunk uploads: Complete in seconds, no timeout ✅ Finalize endpoint: Returns immediately (async), no timeout ✅ Status polling: Each poll completes in milliseconds, no timeout ✅ Total solution: Works with standard 30-second nginx timeout!