Add frontend
This commit is contained in:
pratik
298
DEPLOYMENT_SCRIPTS.md
Normal file
298
DEPLOYMENT_SCRIPTS.md
Normal file
@@ -0,0 +1,298 @@
|
||||
# CF Deployer - Deployment Scripts Documentation
|
||||
|
||||
## Overview
|
||||
|
||||
This repository contains a Spring Boot application for deploying JAR files to Cloud Foundry/Tanzu environments using chunked uploads. There are two deployment scripts designed for different network paths.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────┐
|
||||
│ deploy-chunked.sh │──────► nginx ──────► Spring Boot App
|
||||
│ (Direct to nginx) │ (multipart endpoint)
|
||||
└─────────────────────┘
|
||||
|
||||
┌─────────────────────────┐
|
||||
│deploy-chunked-simple.sh │──► Java Proxy ──► nginx ──► Spring Boot App
|
||||
│ (Through Java proxy) │ (adds headers) (base64 endpoint)
|
||||
└─────────────────────────┘
|
||||
```
|
||||
|
||||
## Deployment Scripts
|
||||
|
||||
### 1. deploy-chunked.sh
|
||||
**Use when**: Direct access to nginx endpoint with cert/key/headers
|
||||
|
||||
**Features**:
|
||||
- Sends chunks as multipart form data (`-F` flags)
|
||||
- Supports client certificates (`--cert`, `--key`)
|
||||
- Supports custom headers (`X-Forwarded-For`, `My-APIM-KEY`)
|
||||
- Uses Spring Boot endpoint: `POST /upload/chunk` (multipart)
|
||||
|
||||
**Configuration**:
|
||||
```bash
|
||||
# Lines 49-55
|
||||
CERT_FILE="/path/to/cert.pem"
|
||||
KEY_FILE="/path/to/key.pem"
|
||||
X_FORWARDED_FOR="192.168.1.100"
|
||||
MY_APIM_KEY="your-api-key"
|
||||
```
|
||||
|
||||
**Curl format**:
|
||||
```bash
|
||||
curl POST /upload/chunk \
|
||||
--cert cert.pem --key key.pem \
|
||||
-H "X-Forwarded-For: ..." \
|
||||
-H "My-APIM-KEY: ..." \
|
||||
-F "uploadSessionId=..." \
|
||||
-F "fileType=..." \
|
||||
-F "chunk=@chunk_file"
|
||||
```
|
||||
|
||||
### 2. deploy-chunked-simple.sh
|
||||
**Use when**: Going through Java proxy that adds cert/headers automatically
|
||||
|
||||
**Features**:
|
||||
- Sends chunks as Base64-encoded text (to work with Java proxy)
|
||||
- Query parameters in URL (for Java proxy's `request.getQueryString()`)
|
||||
- No cert/key/headers needed (Java proxy adds them)
|
||||
- Uses Spring Boot endpoint: `POST /upload/chunk` (text/plain, base64)
|
||||
|
||||
**Configuration**:
|
||||
```bash
|
||||
# Line 24
|
||||
API_BASE="https://myapp.com/v1/utility"
|
||||
```
|
||||
|
||||
**Curl format**:
|
||||
```bash
|
||||
curl POST /upload/chunk?uploadSessionId=...&fileType=... \
|
||||
-H "Content-Type: text/plain" \
|
||||
-H "X-Chunk-Encoding: base64" \
|
||||
-d @base64_chunk_file
|
||||
```
|
||||
|
||||
## Why Two Different Scripts?
|
||||
|
||||
### The Java Proxy Problem
|
||||
|
||||
The Java proxy that sits in front of the Spring Boot app reads the request body as a String:
|
||||
|
||||
```java
|
||||
@RequestBody(required = false) String body
|
||||
```
|
||||
|
||||
**Problem**: Binary multipart data gets corrupted when read as String
|
||||
**Solution**: Base64 encode chunks as text before sending through the proxy
|
||||
|
||||
### Deploy Script Comparison
|
||||
|
||||
| Feature | deploy-chunked.sh | deploy-chunked-simple.sh |
|
||||
|---------|-------------------|--------------------------|
|
||||
| Network Path | Direct to nginx | Through Java proxy |
|
||||
| Chunk Format | Multipart binary | Base64 text |
|
||||
| Query Params | No (uses `-F` form fields) | Yes (in URL) |
|
||||
| Cert/Key | Required in script | Added by Java proxy |
|
||||
| Headers | Required in script | Added by Java proxy |
|
||||
| Spring Endpoint | multipart/form-data | text/plain |
|
||||
|
||||
## Spring Boot Endpoints
|
||||
|
||||
The Spring Boot app has **three** chunk upload endpoints:
|
||||
|
||||
### 1. Multipart Endpoint (Original)
|
||||
```java
|
||||
@PostMapping("/upload/chunk")
|
||||
// Consumes: multipart/form-data
|
||||
// Parameters: All as form fields (-F)
|
||||
// File: @RequestPart("chunk") MultipartFile
|
||||
```
|
||||
**Used by**: deploy-chunked.sh (direct to nginx)
|
||||
|
||||
### 2. Raw Binary Endpoint
|
||||
```java
|
||||
@PostMapping(value = "/upload/chunk", consumes = "application/octet-stream")
|
||||
// Consumes: application/octet-stream
|
||||
// Parameters: Query params in URL
|
||||
// File: @RequestBody byte[]
|
||||
```
|
||||
**Used by**: Not currently used (would fail through Java proxy)
|
||||
|
||||
### 3. Base64 Text Endpoint
|
||||
```java
|
||||
@PostMapping(value = "/upload/chunk", consumes = "text/plain")
|
||||
// Consumes: text/plain
|
||||
// Parameters: Query params in URL
|
||||
// File: @RequestBody String (Base64 decoded)
|
||||
// Header: X-Chunk-Encoding: base64
|
||||
```
|
||||
**Used by**: deploy-chunked-simple.sh (through Java proxy)
|
||||
|
||||
Spring Boot routes to the correct endpoint based on the `Content-Type` header!
|
||||
|
||||
## Common Configuration (Both Scripts)
|
||||
|
||||
Both scripts share these configuration options:
|
||||
|
||||
```bash
|
||||
# Files to deploy
|
||||
JAR_FILE="./app.jar"
|
||||
MANIFEST_FILE="./manifest.yml"
|
||||
|
||||
# Chunk size (1MB recommended for Tanzu)
|
||||
CHUNK_SIZE=1048576
|
||||
|
||||
# Cloud Foundry configuration
|
||||
CF_API_ENDPOINT="https://api.cf.example.com"
|
||||
CF_USERNAME="your-username"
|
||||
CF_PASSWORD="your-password"
|
||||
CF_ORGANIZATION="your-org"
|
||||
CF_SPACE="your-space"
|
||||
CF_APP_NAME="your-app"
|
||||
CF_SKIP_SSL="false"
|
||||
|
||||
# Polling configuration
|
||||
POLL_INTERVAL=5
|
||||
MAX_WAIT=600
|
||||
|
||||
# Debug mode
|
||||
DEBUG_MODE="false" # Set to "true" for verbose output
|
||||
```
|
||||
|
||||
## Deployment Flow
|
||||
|
||||
Both scripts follow the same 5-step process:
|
||||
|
||||
1. **Initialize Upload Session**: POST `/upload/init` with CF credentials
|
||||
2. **Upload JAR Chunks**: POST `/upload/chunk` for each chunk
|
||||
3. **Upload Manifest Chunks**: POST `/upload/chunk` for manifest.yml
|
||||
4. **Finalize Upload**: POST `/upload/finalize?uploadSessionId=...&async=true`
|
||||
5. **Poll Deployment Status**: GET `/deployment/status/{uploadSessionId}`
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Required part 'chunk' is not present"
|
||||
- **Cause**: Nginx stripped multipart body or wrong Content-Type
|
||||
- **Solution**: Use deploy-chunked-simple.sh with Base64 encoding
|
||||
|
||||
### "504 Gateway Timeout" on chunk upload
|
||||
- **Cause**: Java proxy trying to read binary data as String
|
||||
- **Solution**: Use Base64 encoding (deploy-chunked-simple.sh)
|
||||
|
||||
### "Argument list too long"
|
||||
- **Cause**: Base64 string passed as command argument instead of file
|
||||
- **Solution**: Already fixed - script writes Base64 to temp file and uses `-d @file`
|
||||
|
||||
### "Missing uploadSessionId parameter"
|
||||
- **Cause**: Nginx or proxy stripping query parameters
|
||||
- **For deploy-chunked.sh**: Parameters should be in form fields (`-F`)
|
||||
- **For deploy-chunked-simple.sh**: Parameters should be in query string (`?uploadSessionId=...`)
|
||||
|
||||
## Technical Notes
|
||||
|
||||
### Why Not Fix the Java Proxy?
|
||||
|
||||
The Java proxy is shared by multiple services, so modifying it could break other applications. Instead, we adapted the deployment script to work with the proxy's limitations.
|
||||
|
||||
### Why Base64 Encoding?
|
||||
|
||||
When the Java proxy reads binary data as `@RequestBody String body`, it:
|
||||
- Corrupts binary data (non-UTF8 bytes)
|
||||
- May hang or timeout on large binary payloads
|
||||
- Cannot properly forward multipart boundaries
|
||||
|
||||
Base64 encoding converts binary to safe ASCII text that the proxy can handle as a String.
|
||||
|
||||
### Why Query Parameters for Simple Script?
|
||||
|
||||
The Java proxy reconstructs the request using:
|
||||
```java
|
||||
String queryParams = request.getQueryString();
|
||||
String completeRequest = WSGURL + req;
|
||||
if (queryParams != null) {
|
||||
completeRequest = completeRequest + "?" + queryParams;
|
||||
}
|
||||
```
|
||||
|
||||
It only forwards query parameters, not form field parameters, so we must use query strings.
|
||||
|
||||
### Performance Impact of Base64
|
||||
|
||||
Base64 encoding increases payload size by ~33%:
|
||||
- 1MB binary chunk → ~1.33MB Base64 text
|
||||
- Adds CPU overhead for encoding/decoding
|
||||
- Acceptable tradeoff for proxy compatibility
|
||||
|
||||
## Testing
|
||||
|
||||
### Test deploy-chunked.sh (Direct to nginx)
|
||||
```bash
|
||||
# Configure cert/key/headers in script
|
||||
vim deploy-chunked.sh
|
||||
|
||||
# Run with debug
|
||||
DEBUG_MODE="true" ./deploy-chunked.sh
|
||||
```
|
||||
|
||||
### Test deploy-chunked-simple.sh (Through proxy)
|
||||
```bash
|
||||
# Configure API base URL
|
||||
vim deploy-chunked-simple.sh
|
||||
|
||||
# Run with debug
|
||||
DEBUG_MODE="true" ./deploy-chunked-simple.sh
|
||||
```
|
||||
|
||||
## Dependencies
|
||||
|
||||
### Shell Requirements
|
||||
- `bash` 4.0+
|
||||
- `curl`
|
||||
- `awk` (replaces `bc` for file size calculation)
|
||||
- `base64` (for deploy-chunked-simple.sh)
|
||||
- `mktemp`
|
||||
- `split`
|
||||
- `stat`
|
||||
|
||||
### Backend Requirements
|
||||
- Spring Boot 3.2.0+
|
||||
- Java 17+
|
||||
- Gradle 8.14
|
||||
|
||||
## File Reference
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `deploy-chunked.sh` | Direct nginx deployment with cert/headers |
|
||||
| `deploy-chunked-simple.sh` | Java proxy deployment with Base64 |
|
||||
| `CfDeployController.java` | REST endpoints (3 chunk upload variants) |
|
||||
| `ChunkedUploadService.java` | Chunk processing (multipart + raw bytes) |
|
||||
| `AsyncDeploymentService.java` | Background deployment execution |
|
||||
|
||||
## Quick Start
|
||||
|
||||
**For direct nginx access**:
|
||||
```bash
|
||||
cp deploy-chunked.sh my-deploy.sh
|
||||
# Edit configuration
|
||||
vim my-deploy.sh
|
||||
# Run
|
||||
./my-deploy.sh
|
||||
```
|
||||
|
||||
**For Java proxy access**:
|
||||
```bash
|
||||
cp deploy-chunked-simple.sh my-deploy.sh
|
||||
# Edit API_BASE
|
||||
vim my-deploy.sh
|
||||
# Run
|
||||
./my-deploy.sh
|
||||
```
|
||||
|
||||
## Support
|
||||
|
||||
For issues or questions:
|
||||
1. Enable `DEBUG_MODE="true"` in the script
|
||||
2. Check the curl commands and responses
|
||||
3. Review Spring Boot application logs
|
||||
4. Verify nginx/proxy logs for request forwarding
|
||||
Reference in New Issue
Block a user