cf-uploader/DEPLOYMENT_SCRIPTS.md

# CF Deployer - Deployment Scripts Documentation

## Overview

This repository contains a Spring Boot application for deploying JAR files to Cloud Foundry/Tanzu environments using chunked uploads. There are two deployment scripts designed for different network paths.

## Architecture

```
┌─────────────────────┐
│ deploy-chunked.sh   │──────► nginx ──────► Spring Boot App
│ (Direct to nginx)   │                      (multipart endpoint)
└─────────────────────┘

┌─────────────────────────┐
│deploy-chunked-simple.sh │──► Java Proxy ──► nginx ──► Spring Boot App
│ (Through Java proxy)    │   (adds headers)           (base64 endpoint)
└─────────────────────────┘
```

## Deployment Scripts

### 1. deploy-chunked.sh
**Use when**: Direct access to nginx endpoint with cert/key/headers

**Features**:
- Sends chunks as multipart form data (`-F` flags)
- Supports client certificates (`--cert`, `--key`)
- Supports custom headers (`X-Forwarded-For`, `My-APIM-KEY`)
- Uses Spring Boot endpoint: `POST /upload/chunk` (multipart)

**Configuration**:
```bash
# Lines 49-55
CERT_FILE="/path/to/cert.pem"
KEY_FILE="/path/to/key.pem"
X_FORWARDED_FOR="192.168.1.100"
MY_APIM_KEY="your-api-key"
```

**Curl format**:
```bash
curl POST /upload/chunk \
  --cert cert.pem --key key.pem \
  -H "X-Forwarded-For: ..." \
  -H "My-APIM-KEY: ..." \
  -F "uploadSessionId=..." \
  -F "fileType=..." \
  -F "chunk=@chunk_file"
```

### 2. deploy-chunked-simple.sh
**Use when**: Going through Java proxy that adds cert/headers automatically

**Features**:
- Sends chunks as Base64-encoded text (to work with Java proxy)
- Query parameters in URL (for Java proxy's `request.getQueryString()`)
- No cert/key/headers needed (Java proxy adds them)
- Uses Spring Boot endpoint: `POST /upload/chunk` (text/plain, base64)

**Configuration**:
```bash
# Line 24
API_BASE="https://myapp.com/v1/utility"
```

**Curl format**:
```bash
curl POST /upload/chunk?uploadSessionId=...&fileType=... \
  -H "Content-Type: text/plain" \
  -H "X-Chunk-Encoding: base64" \
  -d @base64_chunk_file
```

## Why Two Different Scripts?

### The Java Proxy Problem

The Java proxy that sits in front of the Spring Boot app reads the request body as a String:

```java
@RequestBody(required = false) String body
```

**Problem**: Binary multipart data gets corrupted when read as String
**Solution**: Base64 encode chunks as text before sending through the proxy

### Deploy Script Comparison

| Feature | deploy-chunked.sh | deploy-chunked-simple.sh |
|---------|-------------------|--------------------------|
| Network Path | Direct to nginx | Through Java proxy |
| Chunk Format | Multipart binary | Base64 text |
| Query Params | No (uses `-F` form fields) | Yes (in URL) |
| Cert/Key | Required in script | Added by Java proxy |
| Headers | Required in script | Added by Java proxy |
| Spring Endpoint | multipart/form-data | text/plain |

## Spring Boot Endpoints

The Spring Boot app has **three** chunk upload endpoints:

### 1. Multipart Endpoint (Original)
```java
@PostMapping("/upload/chunk")
// Consumes: multipart/form-data
// Parameters: All as form fields (-F)
// File: @RequestPart("chunk") MultipartFile
```
**Used by**: deploy-chunked.sh (direct to nginx)

### 2. Raw Binary Endpoint
```java
@PostMapping(value = "/upload/chunk", consumes = "application/octet-stream")
// Consumes: application/octet-stream
// Parameters: Query params in URL
// File: @RequestBody byte[]
```
**Used by**: Not currently used (would fail through Java proxy)

### 3. Base64 Text Endpoint
```java
@PostMapping(value = "/upload/chunk", consumes = "text/plain")
// Consumes: text/plain
// Parameters: Query params in URL
// File: @RequestBody String (Base64 decoded)
// Header: X-Chunk-Encoding: base64
```
**Used by**: deploy-chunked-simple.sh (through Java proxy)

Spring Boot routes to the correct endpoint based on the `Content-Type` header!

## Common Configuration (Both Scripts)

Both scripts share these configuration options:

```bash
# Files to deploy
JAR_FILE="./app.jar"
MANIFEST_FILE="./manifest.yml"

# Chunk size (1MB recommended for Tanzu)
CHUNK_SIZE=1048576

# Cloud Foundry configuration
CF_API_ENDPOINT="https://api.cf.example.com"
CF_USERNAME="your-username"
CF_PASSWORD="your-password"
CF_ORGANIZATION="your-org"
CF_SPACE="your-space"
CF_APP_NAME="your-app"
CF_SKIP_SSL="false"

# Polling configuration
POLL_INTERVAL=5
MAX_WAIT=600

# Debug mode
DEBUG_MODE="false"  # Set to "true" for verbose output
```

## Deployment Flow

Both scripts follow the same 5-step process:

1. **Initialize Upload Session**: POST `/upload/init` with CF credentials
2. **Upload JAR Chunks**: POST `/upload/chunk` for each chunk
3. **Upload Manifest Chunks**: POST `/upload/chunk` for manifest.yml
4. **Finalize Upload**: POST `/upload/finalize?uploadSessionId=...&async=true`
5. **Poll Deployment Status**: GET `/deployment/status/{uploadSessionId}`

## Troubleshooting

### "Required part 'chunk' is not present"
- **Cause**: Nginx stripped multipart body or wrong Content-Type
- **Solution**: Use deploy-chunked-simple.sh with Base64 encoding

### "504 Gateway Timeout" on chunk upload
- **Cause**: Java proxy trying to read binary data as String
- **Solution**: Use Base64 encoding (deploy-chunked-simple.sh)

### "Argument list too long"
- **Cause**: Base64 string passed as command argument instead of file
- **Solution**: Already fixed - script writes Base64 to temp file and uses `-d @file`

### "Missing uploadSessionId parameter"
- **Cause**: Nginx or proxy stripping query parameters
- **For deploy-chunked.sh**: Parameters should be in form fields (`-F`)
- **For deploy-chunked-simple.sh**: Parameters should be in query string (`?uploadSessionId=...`)

## Technical Notes

### Why Not Fix the Java Proxy?

The Java proxy is shared by multiple services, so modifying it could break other applications. Instead, we adapted the deployment script to work with the proxy's limitations.

### Why Base64 Encoding?

When the Java proxy reads binary data as `@RequestBody String body`, it:
- Corrupts binary data (non-UTF8 bytes)
- May hang or timeout on large binary payloads
- Cannot properly forward multipart boundaries

Base64 encoding converts binary to safe ASCII text that the proxy can handle as a String.

### Why Query Parameters for Simple Script?

The Java proxy reconstructs the request using:
```java
String queryParams = request.getQueryString();
String completeRequest = WSGURL + req;
if (queryParams != null) {
    completeRequest = completeRequest + "?" + queryParams;
}
```

It only forwards query parameters, not form field parameters, so we must use query strings.

### Performance Impact of Base64

Base64 encoding increases payload size by ~33%:
- 1MB binary chunk → ~1.33MB Base64 text
- Adds CPU overhead for encoding/decoding
- Acceptable tradeoff for proxy compatibility

## Testing

### Test deploy-chunked.sh (Direct to nginx)
```bash
# Configure cert/key/headers in script
vim deploy-chunked.sh

# Run with debug
DEBUG_MODE="true" ./deploy-chunked.sh
```

### Test deploy-chunked-simple.sh (Through proxy)
```bash
# Configure API base URL
vim deploy-chunked-simple.sh

# Run with debug
DEBUG_MODE="true" ./deploy-chunked-simple.sh
```

## Dependencies

### Shell Requirements
- `bash` 4.0+
- `curl`
- `awk` (replaces `bc` for file size calculation)
- `base64` (for deploy-chunked-simple.sh)
- `mktemp`
- `split`
- `stat`

### Backend Requirements
- Spring Boot 3.2.0+
- Java 17+
- Gradle 8.14

## File Reference

| File | Purpose |
|------|---------|
| `deploy-chunked.sh` | Direct nginx deployment with cert/headers |
| `deploy-chunked-simple.sh` | Java proxy deployment with Base64 |
| `CfDeployController.java` | REST endpoints (3 chunk upload variants) |
| `ChunkedUploadService.java` | Chunk processing (multipart + raw bytes) |
| `AsyncDeploymentService.java` | Background deployment execution |

## Quick Start

**For direct nginx access**:
```bash
cp deploy-chunked.sh my-deploy.sh
# Edit configuration
vim my-deploy.sh
# Run
./my-deploy.sh
```

**For Java proxy access**:
```bash
cp deploy-chunked-simple.sh my-deploy.sh
# Edit API_BASE
vim my-deploy.sh
# Run
./my-deploy.sh
```

## Support

For issues or questions:
1. Enable `DEBUG_MODE="true"` in the script
2. Check the curl commands and responses
3. Review Spring Boot application logs
4. Verify nginx/proxy logs for request forwarding