diff --git a/HELM-DEPLOYMENT.md b/HELM-DEPLOYMENT.md new file mode 100644 index 0000000..d20b1b2 --- /dev/null +++ b/HELM-DEPLOYMENT.md @@ -0,0 +1,517 @@ +# Warehouse13 - Kubernetes Deployment with Helm + +This guide covers deploying Warehouse13 to Kubernetes using the official Helm chart. + +## Table of Contents + +1. [Prerequisites](#prerequisites) +2. [Quick Start](#quick-start) +3. [Deployment Scenarios](#deployment-scenarios) +4. [Configuration](#configuration) +5. [Post-Deployment](#post-deployment) +6. [Upgrading](#upgrading) +7. [Troubleshooting](#troubleshooting) + +## Prerequisites + +- Kubernetes 1.19+ cluster +- Helm 3.0+ +- kubectl configured to access your cluster +- Persistent volume provisioner (for production deployments) + +### Installing Helm + +```bash +# macOS +brew install helm + +# Linux +curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash + +# Windows +choco install kubernetes-helm +``` + +## Quick Start + +### 1. Standard Deployment (Internet Access) + +```bash +# Create namespace +kubectl create namespace warehouse13 + +# Install with default values +helm install warehouse13 ./helm/warehouse13 \ + --namespace warehouse13 + +# Wait for pods to be ready +kubectl wait --for=condition=ready pod \ + --all --namespace warehouse13 --timeout=300s +``` + +### 2. Access the Application + +```bash +# Frontend +kubectl port-forward -n warehouse13 svc/warehouse13-frontend 4200:80 + +# API +kubectl port-forward -n warehouse13 svc/warehouse13-api 8000:8000 + +# MinIO Console +kubectl port-forward -n warehouse13 svc/warehouse13-minio 9001:9001 +``` + +Then visit: +- Frontend: http://localhost:4200 +- API Docs: http://localhost:8000/docs +- MinIO Console: http://localhost:9001 + +## Deployment Scenarios + +### Development Environment + +For local testing or CI/CD: + +```bash +helm install warehouse13-dev ./helm/warehouse13 \ + --namespace warehouse13-dev \ + --create-namespace \ + --values ./helm/warehouse13/values-dev.yaml +``` + +**Features:** +- Single replica for all services +- emptyDir storage (no persistence) +- Minimal resource requests +- Always pull latest dev images + +### Production Environment + +For production with ingress and high availability: + +```bash +# First, update the values file with your domain and secrets +cp ./helm/warehouse13/values-production.yaml ./my-production-values.yaml + +# Edit the file: +# - Set postgres.auth.password +# - Set minio.auth.rootUser and rootPassword +# - Set ingress.hosts[0].host to your domain +# - Update storageClass for your environment + +# Install +helm install warehouse13 ./helm/warehouse13 \ + --namespace warehouse13 \ + --create-namespace \ + --values ./my-production-values.yaml +``` + +**Features:** +- 3 replicas for API and frontend +- Persistent storage with PVCs +- Ingress with TLS support +- Resource limits and requests +- Health checks enabled +- Pod anti-affinity for distribution + +### Air-Gapped Environment + +For restricted/disconnected environments: + +```bash +# 1. First, push images to your internal registry +# Example using harbor.internal.example.com + +# Pull images (on internet-connected machine) +docker pull postgres:15-alpine +docker pull minio/minio:latest +docker pull warehouse13/api:v1.0.0 +docker pull warehouse13/frontend:v1.0.0 + +# Tag for internal registry +docker tag postgres:15-alpine harbor.internal.example.com/library/postgres:15-alpine +docker tag minio/minio:latest harbor.internal.example.com/library/minio:latest +docker tag warehouse13/api:v1.0.0 harbor.internal.example.com/warehouse13/api:v1.0.0 +docker tag warehouse13/frontend:v1.0.0 harbor.internal.example.com/warehouse13/frontend:v1.0.0 + +# Push to internal registry +docker push harbor.internal.example.com/library/postgres:15-alpine +docker push harbor.internal.example.com/library/minio:latest +docker push harbor.internal.example.com/warehouse13/api:v1.0.0 +docker push harbor.internal.example.com/warehouse13/frontend:v1.0.0 + +# 2. Update the values file with your registry +cp ./helm/warehouse13/values-airgapped.yaml ./my-airgapped-values.yaml + +# Edit to match your environment: +# - Update all image.repository values +# - Set secure passwords +# - Configure storage classes +# - Add node selectors/tolerations if needed + +# 3. Install on air-gapped cluster +helm install warehouse13 ./helm/warehouse13 \ + --namespace warehouse13 \ + --create-namespace \ + --values ./my-airgapped-values.yaml +``` + +**Features:** +- All images from custom registry +- Local storage class support +- Node selectors for specific nodes +- Tolerations for tainted nodes + +## Configuration + +### Configurable Images + +All component images can be customized: + +```yaml +# PostgreSQL +postgres: + image: + repository: postgres # or your-registry/postgres + tag: 15-alpine + pullPolicy: IfNotPresent + +# MinIO +minio: + image: + repository: minio/minio # or your-registry/minio + tag: latest + pullPolicy: IfNotPresent + +# API Backend +api: + image: + repository: warehouse13/api # or your-registry/warehouse13-api + tag: v1.0.0 + pullPolicy: IfNotPresent + +# Frontend +frontend: + image: + repository: warehouse13/frontend # or your-registry/warehouse13-frontend + tag: v1.0.0 + pullPolicy: IfNotPresent +``` + +### Quick Image Override + +```bash +# Override images from command line +helm install warehouse13 ./helm/warehouse13 \ + --set postgres.image.repository=myregistry.com/postgres \ + --set postgres.image.tag=15-alpine \ + --set minio.image.repository=myregistry.com/minio \ + --set minio.image.tag=latest \ + --set api.image.repository=myregistry.com/warehouse13-api \ + --set api.image.tag=v1.0.0 \ + --set frontend.image.repository=myregistry.com/warehouse13-frontend \ + --set frontend.image.tag=v1.0.0 +``` + +### Storage Configuration + +```yaml +# PostgreSQL storage +postgres: + persistence: + enabled: true + size: 50Gi + storageClass: "fast-ssd" # or "" for default + +# MinIO storage +minio: + persistence: + enabled: true + size: 500Gi + storageClass: "bulk-storage" # or "" for default +``` + +### Resource Configuration + +```yaml +# API resources +api: + resources: + requests: + memory: "512Mi" + cpu: "500m" + limits: + memory: "1Gi" + cpu: "1000m" + +# Frontend resources +frontend: + resources: + requests: + memory: "256Mi" + cpu: "250m" + limits: + memory: "512Mi" + cpu: "500m" +``` + +### Ingress Configuration + +```yaml +ingress: + enabled: true + className: "nginx" + annotations: + cert-manager.io/cluster-issuer: "letsencrypt-prod" + nginx.ingress.kubernetes.io/ssl-redirect: "true" + hosts: + - host: warehouse13.example.com + paths: + - path: / + pathType: Prefix + backend: frontend + - path: /api + pathType: Prefix + backend: api + tls: + - secretName: warehouse13-tls + hosts: + - warehouse13.example.com +``` + +## Post-Deployment + +### Verify Installation + +```bash +# Check all pods are running +kubectl get pods -n warehouse13 + +# Check services +kubectl get svc -n warehouse13 + +# Check PVCs +kubectl get pvc -n warehouse13 + +# Check ingress (if enabled) +kubectl get ingress -n warehouse13 +``` + +### View Logs + +```bash +# API logs +kubectl logs -n warehouse13 -l app.kubernetes.io/component=api --tail=100 -f + +# Frontend logs +kubectl logs -n warehouse13 -l app.kubernetes.io/component=frontend --tail=100 -f + +# PostgreSQL logs +kubectl logs -n warehouse13 warehouse13-postgres-0 --tail=100 -f + +# MinIO logs +kubectl logs -n warehouse13 warehouse13-minio-0 --tail=100 -f +``` + +### Initialize MinIO Bucket + +```bash +# Port-forward to MinIO console +kubectl port-forward -n warehouse13 svc/warehouse13-minio 9001:9001 + +# Open http://localhost:9001 +# Login with credentials from values.yaml +# Create bucket: "artifacts" +``` + +## Upgrading + +### Upgrade to New Version + +```bash +# Update image tags in values file +# Then run upgrade +helm upgrade warehouse13 ./helm/warehouse13 \ + --namespace warehouse13 \ + --values ./my-production-values.yaml \ + --wait \ + --timeout 10m + +# Check rollout status +kubectl rollout status deployment/warehouse13-api -n warehouse13 +kubectl rollout status deployment/warehouse13-frontend -n warehouse13 +``` + +### Rollback + +```bash +# View revision history +helm history warehouse13 -n warehouse13 + +# Rollback to previous version +helm rollback warehouse13 -n warehouse13 + +# Rollback to specific revision +helm rollback warehouse13 2 -n warehouse13 +``` + +### Update Values Only + +```bash +# Update configuration without changing images +helm upgrade warehouse13 ./helm/warehouse13 \ + --namespace warehouse13 \ + --values ./my-updated-values.yaml \ + --reuse-values +``` + +## Backup and Restore + +### PostgreSQL Backup + +```bash +# Create backup +kubectl exec -n warehouse13 warehouse13-postgres-0 -- \ + pg_dump -U warehouse13user warehouse13 > backup-$(date +%Y%m%d).sql + +# Restore +cat backup-20241016.sql | kubectl exec -i -n warehouse13 warehouse13-postgres-0 -- \ + psql -U warehouse13user warehouse13 +``` + +### MinIO Backup + +```bash +# Install MinIO Client +wget https://dl.min.io/client/mc/release/linux-amd64/mc +chmod +x mc + +# Configure +kubectl port-forward -n warehouse13 svc/warehouse13-minio 9000:9000 +mc alias set w13 http://localhost:9000 + +# Backup bucket +mc mirror w13/artifacts ./backup/artifacts-$(date +%Y%m%d) + +# Restore +mc mirror ./backup/artifacts-20241016 w13/artifacts +``` + +### Full Backup + +```bash +# Backup all PVCs +for pvc in $(kubectl get pvc -n warehouse13 -o name); do + pvc_name=$(basename $pvc) + kubectl get -n warehouse13 $pvc -o yaml > backup-${pvc_name}.yaml +done + +# Backup Helm values +helm get values warehouse13 -n warehouse13 > backup-values.yaml +``` + +## Troubleshooting + +### Pods Not Starting + +```bash +# Check pod status +kubectl get pods -n warehouse13 + +# Describe pod for events +kubectl describe pod -n warehouse13 + +# Check logs +kubectl logs -n warehouse13 + +# Common issues: +# - ImagePullBackOff: Check image repository and credentials +# - Pending: Check PVC status and node resources +# - CrashLoopBackOff: Check application logs +``` + +### PVC Issues + +```bash +# Check PVC status +kubectl get pvc -n warehouse13 + +# Describe PVC +kubectl describe pvc -n warehouse13 + +# Common issues: +# - Pending: No storage class or insufficient storage +# - Bound: PVC is healthy +``` + +### Database Connection Issues + +```bash +# Test PostgreSQL connection +kubectl exec -it -n warehouse13 warehouse13-postgres-0 -- \ + psql -U warehouse13user -d warehouse13 + +# Check database logs +kubectl logs -n warehouse13 warehouse13-postgres-0 --tail=100 + +# Verify secret +kubectl get secret -n warehouse13 warehouse13-secrets -o yaml +``` + +### Ingress Not Working + +```bash +# Check ingress status +kubectl get ingress -n warehouse13 +kubectl describe ingress -n warehouse13 warehouse13-ingress + +# Check ingress controller logs +kubectl logs -n ingress-nginx -l app.kubernetes.io/component=controller + +# Verify TLS certificate +kubectl get certificate -n warehouse13 +kubectl describe certificate -n warehouse13 warehouse13-tls +``` + +### Performance Issues + +```bash +# Check resource usage +kubectl top pods -n warehouse13 +kubectl top nodes + +# Check if pods are being throttled +kubectl describe pod -n warehouse13 | grep -A 5 "State:" + +# Increase resources +helm upgrade warehouse13 ./helm/warehouse13 \ + --set api.resources.limits.memory=2Gi \ + --set api.resources.limits.cpu=2000m +``` + +## Uninstalling + +```bash +# Uninstall the release +helm uninstall warehouse13 -n warehouse13 + +# Delete PVCs (data will be lost!) +kubectl delete pvc -n warehouse13 -l app.kubernetes.io/instance=warehouse13 + +# Delete namespace +kubectl delete namespace warehouse13 +``` + +## Additional Resources + +- [Helm Chart README](./helm/warehouse13/README.md) +- [Values Documentation](./helm/warehouse13/values.yaml) +- [Docker Deployment Guide](./DEPLOYMENT.md) +- [Main README](./README.md) + +## Support + +For issues and questions: +- GitHub Issues: https://github.com/yourusername/warehouse13/issues +- Helm Chart Issues: Tag with `helm` label diff --git a/README.md b/README.md index 2c37805..01d2202 100644 --- a/README.md +++ b/README.md @@ -214,35 +214,54 @@ MINIO_BUCKET_NAME=test-artifacts ### Kubernetes with Helm -1. Build and push Docker image: +**Quick Start:** ```bash -docker build -t your-registry/datalake:latest . -docker push your-registry/datalake:latest +helm install warehouse13 ./helm/warehouse13 --namespace warehouse13 --create-namespace ``` -2. Install with Helm: +**Production Deployment:** ```bash -helm install datalake ./helm \ - --set image.repository=your-registry/datalake \ - --set image.tag=latest \ - --namespace datalake \ - --create-namespace +helm install warehouse13 ./helm/warehouse13 \ + --namespace warehouse13 \ + --create-namespace \ + --values ./helm/warehouse13/values-production.yaml ``` -3. Access the API: +**Air-Gapped Deployment:** ```bash -kubectl port-forward -n datalake svc/datalake 8000:8000 +helm install warehouse13 ./helm/warehouse13 \ + --namespace warehouse13 \ + --create-namespace \ + --values ./helm/warehouse13/values-airgapped.yaml ``` +**Access the Application:** +```bash +kubectl port-forward -n warehouse13 svc/warehouse13-frontend 4200:80 +kubectl port-forward -n warehouse13 svc/warehouse13-api 8000:8000 +``` + +### Helm Documentation + +- **Full Helm Guide:** [HELM-DEPLOYMENT.md](./HELM-DEPLOYMENT.md) +- **Chart README:** [helm/warehouse13/README.md](./helm/warehouse13/README.md) +- **Quick Start:** [helm/warehouse13/QUICKSTART.md](./helm/warehouse13/QUICKSTART.md) +- **Example Configurations:** + - Development: [values-dev.yaml](./helm/warehouse13/values-dev.yaml) + - Production: [values-production.yaml](./helm/warehouse13/values-production.yaml) + - Air-Gapped: [values-airgapped.yaml](./helm/warehouse13/values-airgapped.yaml) + ### Helm Configuration -Edit `helm/values.yaml` to customize: -- Replica count -- Resource limits -- Storage backend (S3 vs MinIO) -- Ingress settings -- PostgreSQL settings -- Autoscaling +All component images are fully configurable in `helm/warehouse13/values.yaml`: +- PostgreSQL image and version +- MinIO image and version +- API image and version +- Frontend image and version +- Resource limits and requests +- Storage backend configuration +- Ingress and TLS settings +- Persistence and storage classes ### GitLab CI/CD diff --git a/helm/Chart.yaml b/helm/Chart.yaml index ce650b4..f93d136 100644 --- a/helm/Chart.yaml +++ b/helm/Chart.yaml @@ -1,6 +1,6 @@ apiVersion: v2 -name: datalake -description: Test Artifact Data Lake - Store and query test artifacts +name: warehouse13 +description: Warehouse13 - Enterprise Test Artifact Storage (Legacy Chart - Use ./warehouse13 instead) type: application version: 1.0.0 appVersion: "1.0.0" @@ -8,6 +8,7 @@ keywords: - testing - artifacts - storage - - data-lake + - datalake +deprecated: true maintainers: - - name: Your Team + - name: Warehouse13 Team diff --git a/helm/README.md b/helm/README.md new file mode 100644 index 0000000..9b35934 --- /dev/null +++ b/helm/README.md @@ -0,0 +1,46 @@ +# Helm Charts + +This directory contains Helm charts for deploying Warehouse13. + +## Current Chart (Recommended) + +**Location:** `./warehouse13/` + +The latest, fully-featured Helm chart with: +- Warehouse13 branding +- Configurable images for all components +- Multiple deployment scenarios (dev, production, air-gapped) +- Comprehensive documentation +- Example values files + +**Usage:** +```bash +helm install warehouse13 ./warehouse13 +``` + +**Documentation:** See [warehouse13/README.md](./warehouse13/README.md) + +## Legacy Chart (Deprecated) + +The files in this root `helm/` directory are from an older version and are marked as deprecated. Please use the `./warehouse13/` chart instead. + +## Migration + +If you're using the old chart, migration is straightforward: + +```bash +# Uninstall old chart +helm uninstall datalake + +# Install new chart +helm install warehouse13 ./warehouse13 + +# Or upgrade in place (if compatible) +helm upgrade datalake ./warehouse13 +``` + +Note: Check your values.yaml configuration and update image repositories, resource limits, and other settings as needed. + +## Quick Start + +See [../HELM-DEPLOYMENT.md](../HELM-DEPLOYMENT.md) for comprehensive deployment guide. diff --git a/helm/warehouse13/.helmignore b/helm/warehouse13/.helmignore new file mode 100644 index 0000000..0e8a0eb --- /dev/null +++ b/helm/warehouse13/.helmignore @@ -0,0 +1,23 @@ +# Patterns to ignore when building packages. +# This supports shell glob matching, relative path matching, and +# negation (prefixed with !). Only one pattern per line. +.DS_Store +# Common VCS dirs +.git/ +.gitignore +.bzr/ +.bzrignore +.hg/ +.hgignore +.svn/ +# Common backup files +*.swp +*.bak +*.tmp +*.orig +*~ +# Various IDEs +.project +.idea/ +*.tmproj +.vscode/ diff --git a/helm/warehouse13/Chart.yaml b/helm/warehouse13/Chart.yaml new file mode 100644 index 0000000..b93bf76 --- /dev/null +++ b/helm/warehouse13/Chart.yaml @@ -0,0 +1,16 @@ +apiVersion: v2 +name: warehouse13 +description: Warehouse13 - Enterprise Test Artifact Storage +type: application +version: 1.0.0 +appVersion: "1.0.0" +keywords: + - testing + - artifacts + - storage + - datalake +maintainers: + - name: Warehouse13 Team +home: https://github.com/yourusername/warehouse13 +sources: + - https://github.com/yourusername/warehouse13 diff --git a/helm/warehouse13/QUICKSTART.md b/helm/warehouse13/QUICKSTART.md new file mode 100644 index 0000000..b70e123 --- /dev/null +++ b/helm/warehouse13/QUICKSTART.md @@ -0,0 +1,148 @@ +# Warehouse13 Helm Chart - Quick Start + +## 5-Minute Deployment + +### Prerequisites Check + +```bash +# Verify Kubernetes cluster access +kubectl cluster-info + +# Verify Helm is installed +helm version + +# Create namespace +kubectl create namespace warehouse13 +``` + +### Deploy with Defaults + +```bash +# Install chart +helm install warehouse13 ./helm/warehouse13 --namespace warehouse13 + +# Wait for ready +kubectl wait --for=condition=ready pod --all -n warehouse13 --timeout=5m +``` + +### Access Application + +```bash +# In separate terminals, run: + +# Terminal 1: Frontend +kubectl port-forward -n warehouse13 svc/warehouse13-frontend 4200:80 + +# Terminal 2: API +kubectl port-forward -n warehouse13 svc/warehouse13-api 8000:8000 + +# Terminal 3: MinIO Console +kubectl port-forward -n warehouse13 svc/warehouse13-minio 9001:9001 +``` + +Then open in browser: +- **Frontend:** http://localhost:4200 +- **API Docs:** http://localhost:8000/docs +- **MinIO Console:** http://localhost:9001 + - Username: `minioadmin` + - Password: `minioadmin` + +## Common Scenarios + +### 1. Development (No Persistence) + +```bash +helm install warehouse13 ./helm/warehouse13 \ + --namespace warehouse13 \ + --values ./helm/warehouse13/values-dev.yaml +``` + +### 2. Production (With Ingress) + +```bash +# Update values-production.yaml with your settings first +helm install warehouse13 ./helm/warehouse13 \ + --namespace warehouse13 \ + --values ./helm/warehouse13/values-production.yaml +``` + +### 3. Air-Gapped (Custom Registry) + +```bash +# Update values-airgapped.yaml with your registry first +helm install warehouse13 ./helm/warehouse13 \ + --namespace warehouse13 \ + --values ./helm/warehouse13/values-airgapped.yaml +``` + +### 4. Custom Image Repository + +```bash +helm install warehouse13 ./helm/warehouse13 \ + --namespace warehouse13 \ + --set postgres.image.repository=myregistry.com/postgres \ + --set minio.image.repository=myregistry.com/minio \ + --set api.image.repository=myregistry.com/warehouse13-api \ + --set frontend.image.repository=myregistry.com/warehouse13-frontend +``` + +## Verify Deployment + +```bash +# Check pods +kubectl get pods -n warehouse13 + +# Check services +kubectl get svc -n warehouse13 + +# View logs +kubectl logs -n warehouse13 -l app.kubernetes.io/component=api --tail=50 + +# Check resource usage +kubectl top pods -n warehouse13 +``` + +## Cleanup + +```bash +# Uninstall release +helm uninstall warehouse13 -n warehouse13 + +# Delete PVCs (data will be lost!) +kubectl delete pvc -n warehouse13 --all + +# Delete namespace +kubectl delete namespace warehouse13 +``` + +## Next Steps + +- **Full Documentation:** [README.md](./README.md) +- **Deployment Guide:** [../../HELM-DEPLOYMENT.md](../../HELM-DEPLOYMENT.md) +- **Configuration Options:** [values.yaml](./values.yaml) +- **Example Configs:** [values-dev.yaml](./values-dev.yaml), [values-production.yaml](./values-production.yaml), [values-airgapped.yaml](./values-airgapped.yaml) + +## Troubleshooting + +### Pods stuck in Pending +```bash +kubectl describe pod -n warehouse13 +# Check: PVC status, node resources, storage classes +``` + +### Image pull errors +```bash +kubectl describe pod -n warehouse13 +# Check: Image repository, credentials, network access +``` + +### Database connection errors +```bash +kubectl logs -n warehouse13 warehouse13-postgres-0 +kubectl get secret -n warehouse13 warehouse13-secrets -o yaml +``` + +## Support + +- GitHub Issues: https://github.com/yourusername/warehouse13/issues +- Documentation: https://warehouse13.example.com/docs diff --git a/helm/warehouse13/README.md b/helm/warehouse13/README.md new file mode 100644 index 0000000..eaf0701 --- /dev/null +++ b/helm/warehouse13/README.md @@ -0,0 +1,441 @@ +# Warehouse13 Helm Chart + +Enterprise Test Artifact Storage - Kubernetes deployment via Helm + +## Overview + +This Helm chart deploys the complete Warehouse13 stack on Kubernetes: + +- **PostgreSQL 15** - Metadata database +- **MinIO** - S3-compatible object storage +- **FastAPI Backend** - REST API server +- **Angular Frontend** - Web UI (nginx-served) + +## Prerequisites + +- Kubernetes 1.19+ +- Helm 3.0+ +- PV provisioner support (for persistent storage) + +## Installation + +### Quick Start + +```bash +# Add the Warehouse13 chart repository (if published) +helm repo add warehouse13 https://charts.warehouse13.example.com +helm repo update + +# Install with default values +helm install my-warehouse13 warehouse13/warehouse13 + +# Or install from local chart +helm install my-warehouse13 ./helm/warehouse13 +``` + +### Custom Installation + +```bash +# Install with custom values +helm install my-warehouse13 ./helm/warehouse13 \ + --set postgres.persistence.size=20Gi \ + --set minio.persistence.size=100Gi \ + --set api.replicas=3 + +# Install in a specific namespace +helm install my-warehouse13 ./helm/warehouse13 \ + --namespace warehouse13 \ + --create-namespace +``` + +## Configuration + +### Configurable Images + +All component images can be customized via values.yaml or command-line flags: + +```yaml +postgres: + image: + repository: postgres + tag: 15-alpine + pullPolicy: IfNotPresent + +minio: + image: + repository: minio/minio + tag: latest + pullPolicy: IfNotPresent + +api: + image: + repository: warehouse13/api + tag: latest + pullPolicy: IfNotPresent + +frontend: + image: + repository: warehouse13/frontend + tag: latest + pullPolicy: IfNotPresent +``` + +**Example: Using custom image registry** + +```bash +helm install my-warehouse13 ./helm/warehouse13 \ + --set postgres.image.repository=myregistry.example.com/postgres \ + --set minio.image.repository=myregistry.example.com/minio \ + --set api.image.repository=myregistry.example.com/warehouse13-api \ + --set frontend.image.repository=myregistry.example.com/warehouse13-frontend +``` + +**Example: Air-gapped deployment with specific tags** + +```bash +helm install my-warehouse13 ./helm/warehouse13 \ + --set postgres.image.repository=harbor.internal/library/postgres \ + --set postgres.image.tag=15-alpine \ + --set minio.image.repository=harbor.internal/library/minio \ + --set minio.image.tag=RELEASE.2024-01-01T00-00-00Z \ + --set api.image.repository=harbor.internal/warehouse13/api \ + --set api.image.tag=v1.0.0 \ + --set frontend.image.repository=harbor.internal/warehouse13/frontend \ + --set frontend.image.tag=v1.0.0 +``` + +### Key Parameters + +| Parameter | Description | Default | +|-----------|-------------|---------| +| `global.deploymentMode` | Deployment mode (standard/airgapped) | `standard` | +| `global.storageBackend` | Storage backend (minio/s3) | `minio` | +| `postgres.persistence.enabled` | Enable PostgreSQL persistence | `true` | +| `postgres.persistence.size` | PostgreSQL PVC size | `10Gi` | +| `postgres.auth.username` | PostgreSQL username | `user` | +| `postgres.auth.password` | PostgreSQL password | `password` | +| `minio.persistence.enabled` | Enable MinIO persistence | `true` | +| `minio.persistence.size` | MinIO PVC size | `50Gi` | +| `minio.auth.rootUser` | MinIO root username | `minioadmin` | +| `minio.auth.rootPassword` | MinIO root password | `minioadmin` | +| `api.replicas` | Number of API replicas | `2` | +| `frontend.replicas` | Number of frontend replicas | `2` | +| `ingress.enabled` | Enable ingress | `false` | +| `ingress.className` | Ingress class name | `nginx` | +| `ingress.hosts` | Ingress hosts configuration | See values.yaml | + +### Example Configurations + +#### Production with Ingress + +```yaml +# values-production.yaml +global: + deploymentMode: "standard" + storageBackend: "minio" + +postgres: + persistence: + size: 50Gi + storageClass: "fast-ssd" + resources: + requests: + memory: "1Gi" + cpu: "500m" + limits: + memory: "2Gi" + cpu: "1000m" + +minio: + persistence: + size: 500Gi + storageClass: "bulk-storage" + resources: + requests: + memory: "2Gi" + cpu: "1000m" + limits: + memory: "4Gi" + cpu: "2000m" + +api: + replicas: 3 + resources: + requests: + memory: "512Mi" + cpu: "500m" + limits: + memory: "1Gi" + cpu: "1000m" + +frontend: + replicas: 3 + +ingress: + enabled: true + className: "nginx" + annotations: + cert-manager.io/cluster-issuer: "letsencrypt-prod" + hosts: + - host: warehouse13.example.com + paths: + - path: / + pathType: Prefix + backend: frontend + - path: /api + pathType: Prefix + backend: api + tls: + - secretName: warehouse13-tls + hosts: + - warehouse13.example.com +``` + +```bash +helm install my-warehouse13 ./helm/warehouse13 -f values-production.yaml +``` + +#### Air-Gapped Environment + +```yaml +# values-airgapped.yaml +global: + deploymentMode: "airgapped" + storageBackend: "minio" + +postgres: + image: + repository: harbor.internal.example.com/library/postgres + tag: 15-alpine + pullPolicy: IfNotPresent + +minio: + image: + repository: harbor.internal.example.com/library/minio + tag: RELEASE.2024-01-01T00-00-00Z + pullPolicy: IfNotPresent + +api: + image: + repository: harbor.internal.example.com/warehouse13/api + tag: v1.0.0 + pullPolicy: IfNotPresent + +frontend: + image: + repository: harbor.internal.example.com/warehouse13/frontend + tag: v1.0.0 + pullPolicy: IfNotPresent +``` + +```bash +helm install my-warehouse13 ./helm/warehouse13 -f values-airgapped.yaml +``` + +#### Development/Testing + +```yaml +# values-dev.yaml +global: + deploymentMode: "standard" + +postgres: + persistence: + enabled: false # Use emptyDir for quick testing + resources: + requests: + memory: "128Mi" + cpu: "100m" + +minio: + persistence: + enabled: false + resources: + requests: + memory: "256Mi" + cpu: "100m" + +api: + replicas: 1 + image: + tag: dev + +frontend: + replicas: 1 + image: + tag: dev +``` + +```bash +helm install my-warehouse13 ./helm/warehouse13 -f values-dev.yaml +``` + +## Accessing the Application + +### Port Forwarding (Development) + +```bash +# Access frontend +kubectl port-forward svc/warehouse13-frontend 4200:80 + +# Access API +kubectl port-forward svc/warehouse13-api 8000:8000 + +# Access MinIO console +kubectl port-forward svc/warehouse13-minio 9001:9001 + +# Then visit: +# - Frontend: http://localhost:4200 +# - API: http://localhost:8000 +# - MinIO Console: http://localhost:9001 +``` + +### Via Ingress (Production) + +If ingress is enabled: +``` +https://warehouse13.example.com +``` + +## Upgrading + +```bash +# Upgrade with new values +helm upgrade my-warehouse13 ./helm/warehouse13 \ + --set api.image.tag=v2.0.0 \ + --set frontend.image.tag=v2.0.0 + +# Upgrade with values file +helm upgrade my-warehouse13 ./helm/warehouse13 -f values-production.yaml + +# Upgrade and wait for completion +helm upgrade my-warehouse13 ./helm/warehouse13 --wait --timeout 10m +``` + +## Uninstalling + +```bash +# Uninstall the release +helm uninstall my-warehouse13 + +# Note: PVCs are not deleted automatically. To delete them: +kubectl delete pvc -l app.kubernetes.io/instance=my-warehouse13 +``` + +## Backup and Restore + +### PostgreSQL Backup + +```bash +# Create backup +kubectl exec -it warehouse13-postgres-0 -- pg_dump -U user datalake > backup.sql + +# Restore +kubectl exec -i warehouse13-postgres-0 -- psql -U user datalake < backup.sql +``` + +### MinIO Backup + +```bash +# Install mc (MinIO Client) +# Configure mc alias +mc alias set w13 http://localhost:9001 minioadmin minioadmin + +# Mirror bucket +mc mirror w13/artifacts ./backup/artifacts + +# Restore +mc mirror ./backup/artifacts w13/artifacts +``` + +## Troubleshooting + +### Check Pod Status + +```bash +kubectl get pods -l app.kubernetes.io/name=warehouse13 +``` + +### View Logs + +```bash +# API logs +kubectl logs -l app.kubernetes.io/component=api -f + +# Frontend logs +kubectl logs -l app.kubernetes.io/component=frontend -f + +# PostgreSQL logs +kubectl logs warehouse13-postgres-0 -f + +# MinIO logs +kubectl logs warehouse13-minio-0 -f +``` + +### Check Services + +```bash +kubectl get svc -l app.kubernetes.io/name=warehouse13 +``` + +### Common Issues + +**Pods stuck in Pending** +- Check PVC status: `kubectl get pvc` +- Verify storage class exists: `kubectl get storageclass` +- Check node resources: `kubectl describe nodes` + +**Database connection errors** +- Verify postgres pod is running: `kubectl get pod warehouse13-postgres-0` +- Check database logs: `kubectl logs warehouse13-postgres-0` +- Verify secret exists: `kubectl get secret warehouse13-secrets` + +**Frontend cannot reach API** +- Check ingress configuration: `kubectl describe ingress warehouse13-ingress` +- Verify API service: `kubectl get svc warehouse13-api` +- Check API pod health: `kubectl get pods -l app.kubernetes.io/component=api` + +## Security Considerations + +### Secrets Management + +**Default credentials are for development only!** In production: + +1. **Use external secrets management:** + ```yaml + # Use sealed-secrets, external-secrets, or similar + postgres: + auth: + username: "{{ .Values.externalSecrets.postgresUser }}" + password: "{{ .Values.externalSecrets.postgresPassword }}" + ``` + +2. **Or create secrets manually:** + ```bash + kubectl create secret generic warehouse13-secrets \ + --from-literal=postgres-username=secure-user \ + --from-literal=postgres-password=secure-password \ + --from-literal=minio-root-user=secure-minio-user \ + --from-literal=minio-root-password=secure-minio-password + + # Then install without default secrets + helm install my-warehouse13 ./helm/warehouse13 --set createSecrets=false + ``` + +3. **Enable TLS:** + ```yaml + ingress: + enabled: true + annotations: + cert-manager.io/cluster-issuer: "letsencrypt-prod" + tls: + - secretName: warehouse13-tls + hosts: + - warehouse13.example.com + ``` + +## Support + +For issues and questions: +- GitHub Issues: https://github.com/yourusername/warehouse13/issues +- Documentation: https://warehouse13.example.com/docs diff --git a/helm/warehouse13/templates/NOTES.txt b/helm/warehouse13/templates/NOTES.txt new file mode 100644 index 0000000..602c65a --- /dev/null +++ b/helm/warehouse13/templates/NOTES.txt @@ -0,0 +1,138 @@ + + _ _ _ _ _____ +| | | | | | / |___ / +| | | | __ _ _ __ ___| |__ ___ _ _ ___ / / |_ \ +| |/\| |/ _` | '__/ _ \ '_ \ / _ \| | | / __|/ / ___) | +\ /\ / (_| | | | __/ | | | (_) | |_| \__ \_/ |____/ + \/ \/ \__,_|_| \___|_| |_|\___/ \__,_|___(_) + +Enterprise Test Artifact Storage has been deployed! + +Chart Name: {{ .Chart.Name }} +Chart Version: {{ .Chart.Version }} +App Version: {{ .Chart.AppVersion }} + +Release Name: {{ .Release.Name }} +Namespace: {{ .Release.Namespace }} + +--- + +DEPLOYMENT INFORMATION: + +{{- if .Values.frontend.enabled }} +Frontend: + Service: warehouse13-frontend + Replicas: {{ .Values.frontend.replicas }} + Image: {{ .Values.frontend.image.repository }}:{{ .Values.frontend.image.tag }} +{{- end }} + +{{- if .Values.api.enabled }} +API: + Service: warehouse13-api + Replicas: {{ .Values.api.replicas }} + Image: {{ .Values.api.image.repository }}:{{ .Values.api.image.tag }} +{{- end }} + +{{- if .Values.postgres.enabled }} +PostgreSQL: + Service: warehouse13-postgres + Image: {{ .Values.postgres.image.repository }}:{{ .Values.postgres.image.tag }} + Persistence: {{ if .Values.postgres.persistence.enabled }}Enabled ({{ .Values.postgres.persistence.size }}){{ else }}Disabled (emptyDir){{ end }} +{{- end }} + +{{- if .Values.minio.enabled }} +MinIO: + Service: warehouse13-minio + Image: {{ .Values.minio.image.repository }}:{{ .Values.minio.image.tag }} + Persistence: {{ if .Values.minio.persistence.enabled }}Enabled ({{ .Values.minio.persistence.size }}){{ else }}Disabled (emptyDir){{ end }} +{{- end }} + +--- + +ACCESSING YOUR APPLICATION: + +{{- if .Values.ingress.enabled }} + +1. Via Ingress: +{{- range .Values.ingress.hosts }} + https://{{ .host }} +{{- end }} + +{{- else }} + +1. Using Port Forwarding: + + # Frontend + kubectl port-forward -n {{ .Release.Namespace }} svc/warehouse13-frontend 4200:80 + Then visit: http://localhost:4200 + + # API + kubectl port-forward -n {{ .Release.Namespace }} svc/warehouse13-api 8000:8000 + Then visit: http://localhost:8000/docs + + # MinIO Console + kubectl port-forward -n {{ .Release.Namespace }} svc/warehouse13-minio 9001:9001 + Then visit: http://localhost:9001 + Username: {{ .Values.minio.auth.rootUser }} + Password: {{ .Values.minio.auth.rootPassword }} + +2. Expose via LoadBalancer or Ingress for external access. + +{{- end }} + +--- + +CHECKING STATUS: + + # View all pods + kubectl get pods -n {{ .Release.Namespace }} -l app.kubernetes.io/instance={{ .Release.Name }} + + # Check services + kubectl get svc -n {{ .Release.Namespace }} -l app.kubernetes.io/instance={{ .Release.Name }} + + # View logs + kubectl logs -n {{ .Release.Namespace }} -l app.kubernetes.io/component=api -f + kubectl logs -n {{ .Release.Namespace }} -l app.kubernetes.io/component=frontend -f + +--- + +UPGRADING: + + helm upgrade {{ .Release.Name }} warehouse13/warehouse13 \ + --namespace {{ .Release.Namespace }} + +--- + +UNINSTALLING: + + helm uninstall {{ .Release.Name }} --namespace {{ .Release.Namespace }} + + # Note: PVCs are retained. To delete them: + kubectl delete pvc -n {{ .Release.Namespace }} -l app.kubernetes.io/instance={{ .Release.Name }} + +--- + +{{- if not .Values.ingress.enabled }} +⚠️ IMPORTANT: Ingress is disabled. Enable it for production use: + --set ingress.enabled=true +{{- end }} + +{{- if eq .Values.postgres.auth.password "password" }} +⚠️ WARNING: Using default PostgreSQL password! + For production, set a secure password: + --set postgres.auth.password=YOUR_SECURE_PASSWORD +{{- end }} + +{{- if eq .Values.minio.auth.rootPassword "minioadmin" }} +⚠️ WARNING: Using default MinIO password! + For production, set a secure password: + --set minio.auth.rootPassword=YOUR_SECURE_PASSWORD +{{- end }} + +--- + +For more information, visit: + Documentation: https://github.com/yourusername/warehouse13 + Issues: https://github.com/yourusername/warehouse13/issues + +Thank you for using Warehouse13! diff --git a/helm/warehouse13/templates/_helpers.tpl b/helm/warehouse13/templates/_helpers.tpl new file mode 100644 index 0000000..9845dad --- /dev/null +++ b/helm/warehouse13/templates/_helpers.tpl @@ -0,0 +1,71 @@ +{{/* +Expand the name of the chart. +*/}} +{{- define "warehouse13.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Create a default fully qualified app name. +*/}} +{{- define "warehouse13.fullname" -}} +{{- if .Values.fullnameOverride }} +{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }} +{{- else }} +{{- $name := default .Chart.Name .Values.nameOverride }} +{{- if contains $name .Release.Name }} +{{- .Release.Name | trunc 63 | trimSuffix "-" }} +{{- else }} +{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }} +{{- end }} +{{- end }} +{{- end }} + +{{/* +Create chart name and version as used by the chart label. +*/}} +{{- define "warehouse13.chart" -}} +{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Common labels +*/}} +{{- define "warehouse13.labels" -}} +helm.sh/chart: {{ include "warehouse13.chart" . }} +{{ include "warehouse13.selectorLabels" . }} +{{- if .Chart.AppVersion }} +app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} +{{- end }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- end }} + +{{/* +Selector labels +*/}} +{{- define "warehouse13.selectorLabels" -}} +app.kubernetes.io/name: {{ include "warehouse13.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end }} + +{{/* +Create the name of the service account to use +*/}} +{{- define "warehouse13.serviceAccountName" -}} +{{- if .Values.serviceAccount.create }} +{{- default (include "warehouse13.fullname" .) .Values.serviceAccount.name }} +{{- else }} +{{- default "default" .Values.serviceAccount.name }} +{{- end }} +{{- end }} + +{{/* +PostgreSQL connection string +*/}} +{{- define "warehouse13.postgresUrl" -}} +{{- if .Values.api.env.databaseUrl }} +{{- .Values.api.env.databaseUrl }} +{{- else }} +{{- printf "postgresql://%s:%s@warehouse13-postgres:%d/%s" .Values.postgres.auth.username .Values.postgres.auth.password (.Values.postgres.service.port | int) .Values.postgres.auth.database }} +{{- end }} +{{- end }} diff --git a/helm/warehouse13/templates/api-deployment.yaml b/helm/warehouse13/templates/api-deployment.yaml new file mode 100644 index 0000000..d8bdb1a --- /dev/null +++ b/helm/warehouse13/templates/api-deployment.yaml @@ -0,0 +1,93 @@ +{{- if .Values.api.enabled }} +apiVersion: apps/v1 +kind: Deployment +metadata: + name: warehouse13-api + labels: + {{- include "warehouse13.labels" . | nindent 4 }} + app.kubernetes.io/component: api +spec: + replicas: {{ .Values.api.replicas }} + selector: + matchLabels: + {{- include "warehouse13.selectorLabels" . | nindent 6 }} + app.kubernetes.io/component: api + template: + metadata: + labels: + {{- include "warehouse13.selectorLabels" . | nindent 8 }} + app.kubernetes.io/component: api + spec: + serviceAccountName: {{ include "warehouse13.serviceAccountName" . }} + securityContext: + {{- toYaml .Values.podSecurityContext | nindent 8 }} + containers: + - name: api + securityContext: + {{- toYaml .Values.securityContext | nindent 10 }} + image: "{{ .Values.api.image.repository }}:{{ .Values.api.image.tag }}" + imagePullPolicy: {{ .Values.api.image.pullPolicy }} + ports: + - name: http + containerPort: 8000 + protocol: TCP + env: + - name: DATABASE_URL + valueFrom: + secretKeyRef: + name: warehouse13-secrets + key: database-url + - name: STORAGE_BACKEND + valueFrom: + configMapKeyRef: + name: warehouse13-config + key: STORAGE_BACKEND + - name: MINIO_ENDPOINT + valueFrom: + configMapKeyRef: + name: warehouse13-config + key: MINIO_ENDPOINT + - name: MINIO_ACCESS_KEY + valueFrom: + secretKeyRef: + name: warehouse13-secrets + key: minio-root-user + - name: MINIO_SECRET_KEY + valueFrom: + secretKeyRef: + name: warehouse13-secrets + key: minio-root-password + - name: DEPLOYMENT_MODE + valueFrom: + configMapKeyRef: + name: warehouse13-config + key: DEPLOYMENT_MODE + resources: + {{- toYaml .Values.api.resources | nindent 10 }} + {{- if .Values.api.healthCheck.enabled }} + livenessProbe: + httpGet: + path: {{ .Values.api.healthCheck.liveness.path }} + port: http + initialDelaySeconds: {{ .Values.api.healthCheck.liveness.initialDelaySeconds }} + periodSeconds: {{ .Values.api.healthCheck.liveness.periodSeconds }} + readinessProbe: + httpGet: + path: {{ .Values.api.healthCheck.readiness.path }} + port: http + initialDelaySeconds: {{ .Values.api.healthCheck.readiness.initialDelaySeconds }} + periodSeconds: {{ .Values.api.healthCheck.readiness.periodSeconds }} + {{- end }} + {{- with .Values.nodeSelector }} + nodeSelector: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.affinity }} + affinity: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.tolerations }} + tolerations: + {{- toYaml . | nindent 8 }} + {{- end }} +{{- end }} diff --git a/helm/warehouse13/templates/api-service.yaml b/helm/warehouse13/templates/api-service.yaml new file mode 100644 index 0000000..34a5144 --- /dev/null +++ b/helm/warehouse13/templates/api-service.yaml @@ -0,0 +1,19 @@ +{{- if .Values.api.enabled }} +apiVersion: v1 +kind: Service +metadata: + name: warehouse13-api + labels: + {{- include "warehouse13.labels" . | nindent 4 }} + app.kubernetes.io/component: api +spec: + type: {{ .Values.api.service.type }} + ports: + - port: {{ .Values.api.service.port }} + targetPort: http + protocol: TCP + name: http + selector: + {{- include "warehouse13.selectorLabels" . | nindent 4 }} + app.kubernetes.io/component: api +{{- end }} diff --git a/helm/warehouse13/templates/configmap.yaml b/helm/warehouse13/templates/configmap.yaml new file mode 100644 index 0000000..77394f3 --- /dev/null +++ b/helm/warehouse13/templates/configmap.yaml @@ -0,0 +1,10 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: warehouse13-config + labels: + {{- include "warehouse13.labels" . | nindent 4 }} +data: + DEPLOYMENT_MODE: {{ .Values.global.deploymentMode | quote }} + STORAGE_BACKEND: {{ .Values.global.storageBackend | quote }} + MINIO_ENDPOINT: {{ printf "warehouse13-minio:%d" (.Values.minio.service.apiPort | int) | quote }} diff --git a/helm/warehouse13/templates/frontend-deployment.yaml b/helm/warehouse13/templates/frontend-deployment.yaml new file mode 100644 index 0000000..7ed9c89 --- /dev/null +++ b/helm/warehouse13/templates/frontend-deployment.yaml @@ -0,0 +1,73 @@ +{{- if .Values.frontend.enabled }} +apiVersion: apps/v1 +kind: Deployment +metadata: + name: warehouse13-frontend + labels: + {{- include "warehouse13.labels" . | nindent 4 }} + app.kubernetes.io/component: frontend +spec: + replicas: {{ .Values.frontend.replicas }} + selector: + matchLabels: + {{- include "warehouse13.selectorLabels" . | nindent 6 }} + app.kubernetes.io/component: frontend + template: + metadata: + labels: + {{- include "warehouse13.selectorLabels" . | nindent 8 }} + app.kubernetes.io/component: frontend + spec: + serviceAccountName: {{ include "warehouse13.serviceAccountName" . }} + securityContext: + {{- toYaml .Values.podSecurityContext | nindent 8 }} + containers: + - name: frontend + securityContext: + {{- toYaml .Values.securityContext | nindent 10 }} + image: "{{ .Values.frontend.image.repository }}:{{ .Values.frontend.image.tag }}" + imagePullPolicy: {{ .Values.frontend.image.pullPolicy }} + ports: + - name: http + containerPort: 80 + protocol: TCP + env: + - name: DEPLOYMENT_MODE + valueFrom: + configMapKeyRef: + name: warehouse13-config + key: DEPLOYMENT_MODE + - name: STORAGE_BACKEND + valueFrom: + configMapKeyRef: + name: warehouse13-config + key: STORAGE_BACKEND + resources: + {{- toYaml .Values.frontend.resources | nindent 10 }} + {{- if .Values.frontend.healthCheck.enabled }} + livenessProbe: + httpGet: + path: {{ .Values.frontend.healthCheck.liveness.path }} + port: http + initialDelaySeconds: {{ .Values.frontend.healthCheck.liveness.initialDelaySeconds }} + periodSeconds: {{ .Values.frontend.healthCheck.liveness.periodSeconds }} + readinessProbe: + httpGet: + path: {{ .Values.frontend.healthCheck.readiness.path }} + port: http + initialDelaySeconds: {{ .Values.frontend.healthCheck.readiness.initialDelaySeconds }} + periodSeconds: {{ .Values.frontend.healthCheck.readiness.periodSeconds }} + {{- end }} + {{- with .Values.nodeSelector }} + nodeSelector: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.affinity }} + affinity: + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.tolerations }} + tolerations: + {{- toYaml . | nindent 8 }} + {{- end }} +{{- end }} diff --git a/helm/warehouse13/templates/frontend-service.yaml b/helm/warehouse13/templates/frontend-service.yaml new file mode 100644 index 0000000..256356c --- /dev/null +++ b/helm/warehouse13/templates/frontend-service.yaml @@ -0,0 +1,19 @@ +{{- if .Values.frontend.enabled }} +apiVersion: v1 +kind: Service +metadata: + name: warehouse13-frontend + labels: + {{- include "warehouse13.labels" . | nindent 4 }} + app.kubernetes.io/component: frontend +spec: + type: {{ .Values.frontend.service.type }} + ports: + - port: {{ .Values.frontend.service.port }} + targetPort: http + protocol: TCP + name: http + selector: + {{- include "warehouse13.selectorLabels" . | nindent 4 }} + app.kubernetes.io/component: frontend +{{- end }} diff --git a/helm/warehouse13/templates/ingress.yaml b/helm/warehouse13/templates/ingress.yaml new file mode 100644 index 0000000..36ba3c6 --- /dev/null +++ b/helm/warehouse13/templates/ingress.yaml @@ -0,0 +1,41 @@ +{{- if .Values.ingress.enabled -}} +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: warehouse13-ingress + labels: + {{- include "warehouse13.labels" . | nindent 4 }} + {{- with .Values.ingress.annotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + {{- if .Values.ingress.className }} + ingressClassName: {{ .Values.ingress.className }} + {{- end }} + {{- if .Values.ingress.tls }} + tls: + {{- range .Values.ingress.tls }} + - hosts: + {{- range .hosts }} + - {{ . | quote }} + {{- end }} + secretName: {{ .secretName }} + {{- end }} + {{- end }} + rules: + {{- range .Values.ingress.hosts }} + - host: {{ .host | quote }} + http: + paths: + {{- range .paths }} + - path: {{ .path }} + pathType: {{ .pathType }} + backend: + service: + name: {{ printf "warehouse13-%s" .backend }} + port: + number: {{ if eq .backend "frontend" }}{{ $.Values.frontend.service.port }}{{ else }}{{ $.Values.api.service.port }}{{ end }} + {{- end }} + {{- end }} +{{- end }} diff --git a/helm/warehouse13/templates/minio-service.yaml b/helm/warehouse13/templates/minio-service.yaml new file mode 100644 index 0000000..e61314f --- /dev/null +++ b/helm/warehouse13/templates/minio-service.yaml @@ -0,0 +1,23 @@ +{{- if .Values.minio.enabled }} +apiVersion: v1 +kind: Service +metadata: + name: warehouse13-minio + labels: + {{- include "warehouse13.labels" . | nindent 4 }} + app.kubernetes.io/component: storage +spec: + type: {{ .Values.minio.service.type }} + ports: + - port: {{ .Values.minio.service.apiPort }} + targetPort: api + protocol: TCP + name: api + - port: {{ .Values.minio.service.consolePort }} + targetPort: console + protocol: TCP + name: console + selector: + {{- include "warehouse13.selectorLabels" . | nindent 4 }} + app.kubernetes.io/component: storage +{{- end }} diff --git a/helm/warehouse13/templates/minio-statefulset.yaml b/helm/warehouse13/templates/minio-statefulset.yaml new file mode 100644 index 0000000..0479960 --- /dev/null +++ b/helm/warehouse13/templates/minio-statefulset.yaml @@ -0,0 +1,87 @@ +{{- if .Values.minio.enabled }} +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: warehouse13-minio + labels: + {{- include "warehouse13.labels" . | nindent 4 }} + app.kubernetes.io/component: storage +spec: + serviceName: warehouse13-minio + replicas: 1 + selector: + matchLabels: + {{- include "warehouse13.selectorLabels" . | nindent 6 }} + app.kubernetes.io/component: storage + template: + metadata: + labels: + {{- include "warehouse13.selectorLabels" . | nindent 8 }} + app.kubernetes.io/component: storage + spec: + serviceAccountName: {{ include "warehouse13.serviceAccountName" . }} + securityContext: + {{- toYaml .Values.podSecurityContext | nindent 8 }} + containers: + - name: minio + image: "{{ .Values.minio.image.repository }}:{{ .Values.minio.image.tag }}" + imagePullPolicy: {{ .Values.minio.image.pullPolicy }} + command: + - minio + - server + - /data + - --console-address + - ":9001" + ports: + - name: api + containerPort: 9000 + protocol: TCP + - name: console + containerPort: 9001 + protocol: TCP + env: + - name: MINIO_ROOT_USER + valueFrom: + secretKeyRef: + name: warehouse13-secrets + key: minio-root-user + - name: MINIO_ROOT_PASSWORD + valueFrom: + secretKeyRef: + name: warehouse13-secrets + key: minio-root-password + volumeMounts: + - name: data + mountPath: /data + resources: + {{- toYaml .Values.minio.resources | nindent 10 }} + livenessProbe: + httpGet: + path: /minio/health/live + port: api + initialDelaySeconds: 30 + periodSeconds: 10 + readinessProbe: + httpGet: + path: /minio/health/ready + port: api + initialDelaySeconds: 10 + periodSeconds: 5 + {{- if .Values.minio.persistence.enabled }} + volumeClaimTemplates: + - metadata: + name: data + spec: + accessModes: [ "ReadWriteOnce" ] + {{- if .Values.minio.persistence.storageClass }} + storageClassName: {{ .Values.minio.persistence.storageClass }} + {{- end }} + resources: + requests: + storage: {{ .Values.minio.persistence.size }} + {{- else }} + volumes: + - name: data + emptyDir: {} + {{- end }} +{{- end }} diff --git a/helm/warehouse13/templates/postgres-service.yaml b/helm/warehouse13/templates/postgres-service.yaml new file mode 100644 index 0000000..987a64d --- /dev/null +++ b/helm/warehouse13/templates/postgres-service.yaml @@ -0,0 +1,19 @@ +{{- if .Values.postgres.enabled }} +apiVersion: v1 +kind: Service +metadata: + name: warehouse13-postgres + labels: + {{- include "warehouse13.labels" . | nindent 4 }} + app.kubernetes.io/component: database +spec: + type: {{ .Values.postgres.service.type }} + ports: + - port: {{ .Values.postgres.service.port }} + targetPort: postgres + protocol: TCP + name: postgres + selector: + {{- include "warehouse13.selectorLabels" . | nindent 4 }} + app.kubernetes.io/component: database +{{- end }} diff --git a/helm/warehouse13/templates/postgres-statefulset.yaml b/helm/warehouse13/templates/postgres-statefulset.yaml new file mode 100644 index 0000000..85cbfee --- /dev/null +++ b/helm/warehouse13/templates/postgres-statefulset.yaml @@ -0,0 +1,89 @@ +{{- if .Values.postgres.enabled }} +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: warehouse13-postgres + labels: + {{- include "warehouse13.labels" . | nindent 4 }} + app.kubernetes.io/component: database +spec: + serviceName: warehouse13-postgres + replicas: 1 + selector: + matchLabels: + {{- include "warehouse13.selectorLabels" . | nindent 6 }} + app.kubernetes.io/component: database + template: + metadata: + labels: + {{- include "warehouse13.selectorLabels" . | nindent 8 }} + app.kubernetes.io/component: database + spec: + serviceAccountName: {{ include "warehouse13.serviceAccountName" . }} + securityContext: + {{- toYaml .Values.podSecurityContext | nindent 8 }} + containers: + - name: postgres + image: "{{ .Values.postgres.image.repository }}:{{ .Values.postgres.image.tag }}" + imagePullPolicy: {{ .Values.postgres.image.pullPolicy }} + ports: + - name: postgres + containerPort: 5432 + protocol: TCP + env: + - name: POSTGRES_USER + valueFrom: + secretKeyRef: + name: warehouse13-secrets + key: postgres-username + - name: POSTGRES_PASSWORD + valueFrom: + secretKeyRef: + name: warehouse13-secrets + key: postgres-password + - name: POSTGRES_DB + valueFrom: + secretKeyRef: + name: warehouse13-secrets + key: postgres-database + - name: PGDATA + value: /var/lib/postgresql/data/pgdata + volumeMounts: + - name: data + mountPath: /var/lib/postgresql/data + resources: + {{- toYaml .Values.postgres.resources | nindent 10 }} + livenessProbe: + exec: + command: + - pg_isready + - -U + - $(POSTGRES_USER) + initialDelaySeconds: 30 + periodSeconds: 10 + readinessProbe: + exec: + command: + - pg_isready + - -U + - $(POSTGRES_USER) + initialDelaySeconds: 10 + periodSeconds: 5 + {{- if .Values.postgres.persistence.enabled }} + volumeClaimTemplates: + - metadata: + name: data + spec: + accessModes: [ "ReadWriteOnce" ] + {{- if .Values.postgres.persistence.storageClass }} + storageClassName: {{ .Values.postgres.persistence.storageClass }} + {{- end }} + resources: + requests: + storage: {{ .Values.postgres.persistence.size }} + {{- else }} + volumes: + - name: data + emptyDir: {} + {{- end }} +{{- end }} diff --git a/helm/warehouse13/templates/secret.yaml b/helm/warehouse13/templates/secret.yaml new file mode 100644 index 0000000..451d0a6 --- /dev/null +++ b/helm/warehouse13/templates/secret.yaml @@ -0,0 +1,14 @@ +apiVersion: v1 +kind: Secret +metadata: + name: warehouse13-secrets + labels: + {{- include "warehouse13.labels" . | nindent 4 }} +type: Opaque +stringData: + postgres-username: {{ .Values.postgres.auth.username | quote }} + postgres-password: {{ .Values.postgres.auth.password | quote }} + postgres-database: {{ .Values.postgres.auth.database | quote }} + minio-root-user: {{ .Values.minio.auth.rootUser | quote }} + minio-root-password: {{ .Values.minio.auth.rootPassword | quote }} + database-url: {{ include "warehouse13.postgresUrl" . | quote }} diff --git a/helm/warehouse13/templates/serviceaccount.yaml b/helm/warehouse13/templates/serviceaccount.yaml new file mode 100644 index 0000000..ecffac5 --- /dev/null +++ b/helm/warehouse13/templates/serviceaccount.yaml @@ -0,0 +1,12 @@ +{{- if .Values.serviceAccount.create -}} +apiVersion: v1 +kind: ServiceAccount +metadata: + name: {{ include "warehouse13.serviceAccountName" . }} + labels: + {{- include "warehouse13.labels" . | nindent 4 }} + {{- with .Values.serviceAccount.annotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +{{- end }} diff --git a/helm/warehouse13/values-airgapped.yaml b/helm/warehouse13/values-airgapped.yaml new file mode 100644 index 0000000..7bea087 --- /dev/null +++ b/helm/warehouse13/values-airgapped.yaml @@ -0,0 +1,99 @@ +# Warehouse13 - Air-Gapped Deployment Example +# Use this for restricted/disconnected environments + +global: + deploymentMode: "airgapped" + storageBackend: "minio" + +# PostgreSQL with custom registry +postgres: + enabled: true + image: + repository: harbor.internal.example.com/library/postgres + tag: 15-alpine + pullPolicy: IfNotPresent + auth: + username: warehouse13user + password: CHANGE_ME_SECURE_PASSWORD + database: warehouse13 + persistence: + enabled: true + size: 20Gi + storageClass: "local-storage" + resources: + requests: + memory: "512Mi" + cpu: "500m" + limits: + memory: "1Gi" + cpu: "1000m" + +# MinIO with custom registry +minio: + enabled: true + image: + repository: harbor.internal.example.com/library/minio + tag: RELEASE.2024-01-01T00-00-00Z + pullPolicy: IfNotPresent + auth: + rootUser: CHANGE_ME_MINIO_USER + rootPassword: CHANGE_ME_MINIO_PASSWORD + persistence: + enabled: true + size: 100Gi + storageClass: "local-storage" + resources: + requests: + memory: "1Gi" + cpu: "500m" + limits: + memory: "2Gi" + cpu: "1000m" + +# API with custom registry +api: + enabled: true + image: + repository: harbor.internal.example.com/warehouse13/api + tag: v1.0.0 + pullPolicy: IfNotPresent + replicas: 2 + resources: + requests: + memory: "512Mi" + cpu: "500m" + limits: + memory: "1Gi" + cpu: "1000m" + +# Frontend with custom registry +frontend: + enabled: true + image: + repository: harbor.internal.example.com/warehouse13/frontend + tag: v1.0.0 + pullPolicy: IfNotPresent + replicas: 2 + resources: + requests: + memory: "256Mi" + cpu: "250m" + limits: + memory: "512Mi" + cpu: "500m" + +# Ingress disabled for air-gapped - use NodePort or port-forward +ingress: + enabled: false + +# Node selector for specific nodes +nodeSelector: + environment: production + storage: local + +# Tolerations for tainted nodes +tolerations: + - key: "airgapped" + operator: "Equal" + value: "true" + effect: "NoSchedule" diff --git a/helm/warehouse13/values-dev.yaml b/helm/warehouse13/values-dev.yaml new file mode 100644 index 0000000..0baf692 --- /dev/null +++ b/helm/warehouse13/values-dev.yaml @@ -0,0 +1,86 @@ +# Warehouse13 - Development/Testing Deployment Example +# Use this for local testing or CI/CD environments + +global: + deploymentMode: "standard" + storageBackend: "minio" + +postgres: + enabled: true + image: + repository: postgres + tag: 15-alpine + pullPolicy: IfNotPresent + auth: + username: dev + password: dev + database: warehouse13dev + persistence: + enabled: false # Use emptyDir for faster cleanup + resources: + requests: + memory: "128Mi" + cpu: "100m" + limits: + memory: "256Mi" + cpu: "250m" + +minio: + enabled: true + image: + repository: minio/minio + tag: latest + pullPolicy: IfNotPresent + auth: + rootUser: minioadmin + rootPassword: minioadmin + persistence: + enabled: false # Use emptyDir for faster cleanup + resources: + requests: + memory: "256Mi" + cpu: "100m" + limits: + memory: "512Mi" + cpu: "250m" + +api: + enabled: true + image: + repository: warehouse13/api + tag: dev + pullPolicy: Always # Always pull latest dev image + replicas: 1 + resources: + requests: + memory: "256Mi" + cpu: "250m" + limits: + memory: "512Mi" + cpu: "500m" + healthCheck: + enabled: true + +frontend: + enabled: true + image: + repository: warehouse13/frontend + tag: dev + pullPolicy: Always # Always pull latest dev image + replicas: 1 + resources: + requests: + memory: "128Mi" + cpu: "100m" + limits: + memory: "256Mi" + cpu: "250m" + healthCheck: + enabled: true + +ingress: + enabled: false # Use port-forward for dev + +serviceAccount: + create: true + name: "warehouse13-dev" diff --git a/helm/warehouse13/values-production.yaml b/helm/warehouse13/values-production.yaml new file mode 100644 index 0000000..d7ab719 --- /dev/null +++ b/helm/warehouse13/values-production.yaml @@ -0,0 +1,118 @@ +# Warehouse13 - Production Deployment Example +# Use this for production environments with ingress and proper resources + +global: + deploymentMode: "standard" + storageBackend: "minio" + +postgres: + enabled: true + image: + repository: postgres + tag: 15-alpine + pullPolicy: IfNotPresent + auth: + username: warehouse13user + password: CHANGE_ME_SECURE_PASSWORD + database: warehouse13 + persistence: + enabled: true + size: 50Gi + storageClass: "fast-ssd" + resources: + requests: + memory: "1Gi" + cpu: "1000m" + limits: + memory: "2Gi" + cpu: "2000m" + +minio: + enabled: true + image: + repository: minio/minio + tag: latest + pullPolicy: IfNotPresent + auth: + rootUser: CHANGE_ME_MINIO_USER + rootPassword: CHANGE_ME_MINIO_PASSWORD + persistence: + enabled: true + size: 500Gi + storageClass: "bulk-storage" + resources: + requests: + memory: "2Gi" + cpu: "1000m" + limits: + memory: "4Gi" + cpu: "2000m" + +api: + enabled: true + image: + repository: warehouse13/api + tag: v1.0.0 + pullPolicy: IfNotPresent + replicas: 3 + resources: + requests: + memory: "512Mi" + cpu: "500m" + limits: + memory: "1Gi" + cpu: "1000m" + healthCheck: + enabled: true + +frontend: + enabled: true + image: + repository: warehouse13/frontend + tag: v1.0.0 + pullPolicy: IfNotPresent + replicas: 3 + resources: + requests: + memory: "256Mi" + cpu: "250m" + limits: + memory: "512Mi" + cpu: "500m" + healthCheck: + enabled: true + +ingress: + enabled: true + className: "nginx" + annotations: + cert-manager.io/cluster-issuer: "letsencrypt-prod" + nginx.ingress.kubernetes.io/ssl-redirect: "true" + nginx.ingress.kubernetes.io/force-ssl-redirect: "true" + hosts: + - host: warehouse13.example.com + paths: + - path: / + pathType: Prefix + backend: frontend + - path: /api + pathType: Prefix + backend: api + tls: + - secretName: warehouse13-tls + hosts: + - warehouse13.example.com + +# Affinity for pod distribution +affinity: + podAntiAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + - weight: 100 + podAffinityTerm: + labelSelector: + matchExpressions: + - key: app.kubernetes.io/name + operator: In + values: + - warehouse13 + topologyKey: kubernetes.io/hostname diff --git a/helm/warehouse13/values.yaml b/helm/warehouse13/values.yaml new file mode 100644 index 0000000..7cd4ce6 --- /dev/null +++ b/helm/warehouse13/values.yaml @@ -0,0 +1,168 @@ +# Warehouse13 - Enterprise Test Artifact Storage +# Default values for Helm chart + +# Global settings +global: + deploymentMode: "standard" # standard or airgapped + storageBackend: "minio" # minio or s3 + +# PostgreSQL Database +postgres: + enabled: true + image: + repository: postgres + tag: 15-alpine + pullPolicy: IfNotPresent + auth: + username: user + password: password + database: datalake + persistence: + enabled: true + size: 10Gi + storageClass: "" + resources: + requests: + memory: "256Mi" + cpu: "250m" + limits: + memory: "512Mi" + cpu: "500m" + service: + type: ClusterIP + port: 5432 + +# MinIO Object Storage +minio: + enabled: true + image: + repository: minio/minio + tag: latest + pullPolicy: IfNotPresent + auth: + rootUser: minioadmin + rootPassword: minioadmin + persistence: + enabled: true + size: 50Gi + storageClass: "" + resources: + requests: + memory: "512Mi" + cpu: "250m" + limits: + memory: "1Gi" + cpu: "1000m" + service: + type: ClusterIP + apiPort: 9000 + consolePort: 9001 + +# API Backend +api: + enabled: true + image: + repository: warehouse13/api + tag: latest + pullPolicy: IfNotPresent + replicas: 2 + env: + databaseUrl: "postgresql://user:password@warehouse13-postgres:5432/datalake" + storageBackend: "minio" + minioEndpoint: "warehouse13-minio:9000" + resources: + requests: + memory: "256Mi" + cpu: "250m" + limits: + memory: "512Mi" + cpu: "500m" + service: + type: ClusterIP + port: 8000 + healthCheck: + enabled: true + liveness: + path: /health + initialDelaySeconds: 30 + periodSeconds: 10 + readiness: + path: /ready + initialDelaySeconds: 10 + periodSeconds: 5 + +# Frontend +frontend: + enabled: true + image: + repository: warehouse13/frontend + tag: latest + pullPolicy: IfNotPresent + replicas: 2 + resources: + requests: + memory: "128Mi" + cpu: "100m" + limits: + memory: "256Mi" + cpu: "250m" + service: + type: ClusterIP + port: 80 + healthCheck: + enabled: true + liveness: + path: / + initialDelaySeconds: 10 + periodSeconds: 10 + readiness: + path: / + initialDelaySeconds: 5 + periodSeconds: 5 + +# Ingress +ingress: + enabled: false + className: "nginx" + annotations: + # cert-manager.io/cluster-issuer: "letsencrypt-prod" + hosts: + - host: warehouse13.example.com + paths: + - path: / + pathType: Prefix + backend: frontend + - path: /api + pathType: Prefix + backend: api + tls: [] + # - secretName: warehouse13-tls + # hosts: + # - warehouse13.example.com + +# Service Account +serviceAccount: + create: true + annotations: {} + name: "warehouse13" + +# Pod Security +podSecurityContext: + fsGroup: 2000 + +securityContext: + capabilities: + drop: + - ALL + readOnlyRootFilesystem: false + runAsNonRoot: true + runAsUser: 1000 + +# Node selector +nodeSelector: {} + +# Tolerations +tolerations: [] + +# Affinity +affinity: {}