31 Commits

Author SHA1 Message Date
Mondo Diaz
ba7cd96107 Add package dependencies and project settings page
Package Dependencies:
- Add artifact dependency management system
- Add dependency API endpoints (get, resolve, reverse)
- Add ensure file parsing for declaring dependencies
- Add circular dependency and conflict detection
- Add frontend dependency visualization with graph modal
- Add migration for artifact_dependencies table

Project Settings Page (#65):
- Add dedicated settings page for project admins
- General settings section (description, visibility)
- Access management section (moved from project page)
- Danger zone with inline delete confirmation
- Add Settings button to project page header
2026-01-27 15:29:51 +00:00
Mondo Diaz
6c8b922818 Merge branch 'fix/reset-stage-before-tests' into 'main'
Add pre-test stage reset to ensure known environment state

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!44
2026-01-26 09:13:04 -06:00
Mondo Diaz
99d28cf9c6 Add pre-test stage reset to ensure known environment state 2026-01-26 09:13:03 -06:00
Dane Moss
b5579f1643 Merge branch 'release_0.5.1' into 'main'
add CL entry to bump version

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!43
2026-01-23 15:37:09 -07:00
Dane Moss
fafa03e4ce add CL entry to bump version 2026-01-23 15:37:09 -07:00
Mondo Diaz
d4b2da3232 Merge branch 'fix/release-wait-for-stage-tests' into 'main'
Add gitleaks fingerprint for test file false positive

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!42
2026-01-23 16:16:03 -06:00
Mondo Diaz
7b04bbdf05 Add gitleaks fingerprint for test file false positive 2026-01-23 16:16:02 -06:00
Mondo Diaz
3a807870a3 Merge branch 'fix/ci-prod-namespace' into 'main'
Fix production CI deployment and simplify tag pipeline

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!41
2026-01-23 15:50:24 -06:00
Mondo Diaz
f966fde7df Fix production CI deployment and simplify tag pipeline 2026-01-23 15:50:24 -06:00
Mondo Diaz
133d9cbfd6 Merge branch 'bump_version' into 'main'
add changelog entry to cut a new release

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!40
2026-01-23 13:00:51 -06:00
Dane Moss
276b4f2743 add changelog entry to cut a new release 2026-01-23 10:46:20 -07:00
Mondo Diaz
67ac6bb3f8 Merge branch 'fix/factory-reset-admin-user' into 'main'
Update CHANGELOG with factory reset fixes (#60)

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!39
2026-01-23 09:33:01 -06:00
Mondo Diaz
b0bb3ed569 Update CHANGELOG with factory reset fixes (#60) 2026-01-21 23:44:45 +00:00
Mondo Diaz
1ac75e1017 Fix factory reset and improve reset_stage CI job
- Add create_default_admin() call to factory reset (admin user wasn't being
  created after reset, only on server restart)
- Add retry logic to reset_stage CI job (3 attempts with 5s delay)
- Use proper context manager for httpx client
- Increase timeout to 120s for reset operation
- Add retry: 1 at job level for transient failures
2026-01-21 23:20:48 +00:00
Mondo Diaz
693613f111 Fix factory reset - capture username before dropping tables 2026-01-21 23:18:29 +00:00
Mondo Diaz
9da4ae8c0d Add gitleaks fingerprint for test file false positive 2026-01-21 22:59:08 +00:00
Mondo Diaz
7ffdc64364 Fix seed_database call in factory reset - pass fresh db session 2026-01-21 22:51:03 +00:00
Mondo Diaz
6abc0c88b0 Merge branch 'feature/stage-reset-job' into 'main'
Fix reset_stage job to read STAGE_URL from environment

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!38
2026-01-21 16:39:39 -06:00
Mondo Diaz
e96dc5cde8 Fix reset_stage job to read STAGE_URL from environment 2026-01-21 22:25:04 +00:00
Mondo Diaz
cba5bac383 Merge branch 'feature/stage-reset-job' into 'main'
Add factory reset endpoint for stage environment cleanup (#54)

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!37
2026-01-21 16:00:02 -06:00
Mondo Diaz
535280a783 Add factory reset endpoint for stage environment cleanup (#54) 2026-01-21 16:00:02 -06:00
Dane Moss
c9026e1950 Merge branch 'fix/s3-irsa-credentials' into 'main'
Fix S3 client to support IRSA credentials (#54)

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!36
2026-01-21 13:42:53 -07:00
Mondo Diaz
fedbd95cf4 Fix S3 client to support IRSA credentials (#54) 2026-01-21 13:42:53 -07:00
Dane Moss
255e25d66d Merge branch 'fix/prod-values-config' into 'main'
Configure prod and stage for AWS services (RDS, S3, Secrets Manager)

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!35
2026-01-21 12:32:44 -07:00
Mondo Diaz
427d2fec70 Configure prod and stage for AWS services (RDS, S3, Secrets Manager) 2026-01-21 12:32:44 -07:00
Mondo Diaz
199821b34d Merge branch 'feature/upload-download-tests' into 'main'
Add comprehensive upload/download tests and streaming enhancements (#38, #40, #42, #43)

Closes #38, #40, #42, and #43

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!34
2026-01-21 09:35:12 -06:00
Mondo Diaz
584acd1e90 Add comprehensive upload/download tests and streaming enhancements (#38, #40, #42, #43) 2026-01-21 09:35:12 -06:00
Mondo Diaz
f7ffc1c877 Merge branch 'feature/separate-version-tag' into 'main'
Add separate version tracking for artifacts

Closes orchard-python-poetry#56

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!33
2026-01-16 11:36:08 -06:00
Mondo Diaz
b93d5a9c68 Add separate version tracking for artifacts 2026-01-16 11:36:08 -06:00
Mondo Diaz
a98ac154d5 Merge branch 'fix/deploy-job-dependencies' into 'main'
Fix deploy job dependencies and add production deployment (#63)

Closes #63

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!32
2026-01-15 15:16:07 -06:00
Mondo Diaz
823dfcb400 Fix deploy job dependencies and add production deployment (#63) 2026-01-15 15:16:07 -06:00
48 changed files with 12282 additions and 585 deletions

View File

@@ -8,6 +8,17 @@ variables:
PROSPER_VERSION: v0.64.1
# Use internal PyPI proxy instead of public internet
PIP_INDEX_URL: https://deps.global.bsf.tools/artifactory/api/pypi/pypi.org/simple
# Environment URLs (used by deploy and test jobs)
STAGE_URL: https://orchard-stage.common.global.bsf.tools
PROD_URL: https://orchard.common.global.bsf.tools
# Stage environment AWS resources (used by reset job)
STAGE_RDS_HOST: orchard-stage.cluster-cvw3jzjkozoc.us-gov-west-1.rds.amazonaws.com
STAGE_RDS_DBNAME: postgres
STAGE_SECRET_ARN: "arn:aws-us-gov:secretsmanager:us-gov-west-1:052673043337:secret:rds!cluster-a573672b-1a38-4665-a654-1b7df37b5297-IaeFQL"
STAGE_S3_BUCKET: orchard-artifacts-stage
AWS_REGION: us-gov-west-1
# Shared pip cache directory
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.pip-cache"
# Prevent duplicate pipelines for MRs
workflow:
@@ -25,15 +36,110 @@ stages:
- analyze
- deploy
# Override Prosper template jobs to exclude tag pipelines
# Tags only run deploy_prod and smoke_test_prod (image already built on main)
build_image:
rules:
- if: '$CI_COMMIT_TAG'
when: never
- when: on_success
test_image:
rules:
- if: '$CI_COMMIT_TAG'
when: never
- when: on_success
hadolint:
rules:
- if: '$CI_COMMIT_TAG'
when: never
- when: on_success
kics:
variables:
KICS_CONFIG: kics.config
rules:
- if: '$CI_COMMIT_TAG'
when: never
- when: on_success
# Post-deployment integration tests template
secrets:
rules:
- if: '$CI_COMMIT_TAG'
when: never
- when: on_success
app_deps_scan:
rules:
- if: '$CI_COMMIT_TAG'
when: never
- when: on_success
cve_scan:
rules:
- if: '$CI_COMMIT_TAG'
when: never
- when: on_success
app_sbom_analysis:
rules:
- if: '$CI_COMMIT_TAG'
when: never
- when: on_success
cve_sbom_analysis:
rules:
- if: '$CI_COMMIT_TAG'
when: never
- when: on_success
# Override release job to wait for stage integration tests before creating tag
# This ensures the tag (which triggers prod deploy) is only created after stage passes
release:
needs: [integration_test_stage, changelog]
# Full integration test suite template (for feature/stage deployments)
# Runs the complete pytest integration test suite against the deployed environment
.integration_test_template: &integration_test_template
stage: deploy # Runs in deploy stage, but after deployment due to 'needs'
image: deps.global.bsf.tools/docker/python:3.12-slim
timeout: 10m
timeout: 20m # Full suite takes longer than smoke tests
interruptible: true # Cancel if new pipeline starts
retry: 1 # Retry once on failure (network flakiness)
cache:
key: pip-$CI_COMMIT_REF_SLUG
paths:
- .pip-cache/
policy: pull-push
before_script:
- pip install --index-url "$PIP_INDEX_URL" -r backend/requirements.txt
- pip install --index-url "$PIP_INDEX_URL" pytest pytest-asyncio httpx
script:
- cd backend
# Run full integration test suite, excluding:
# - large/slow tests
# - requires_direct_s3 tests (can't access MinIO from outside K8s cluster)
# ORCHARD_TEST_URL tells the tests which server to connect to
# Note: Auth tests work because dev/stage deployments have relaxed rate limits
- |
python -m pytest tests/integration/ -v \
--junitxml=integration-report.xml \
-m "not large and not slow and not requires_direct_s3" \
--tb=short
artifacts:
when: always
expire_in: 1 week
paths:
- backend/integration-report.xml
reports:
junit: backend/integration-report.xml
# Lightweight smoke test template (for production - no test data creation)
.smoke_test_template: &smoke_test_template
stage: deploy
image: deps.global.bsf.tools/docker/python:3.12-slim
timeout: 5m
before_script:
- pip install --index-url "$PIP_INDEX_URL" httpx
script:
@@ -43,12 +149,12 @@ kics:
import os
import sys
BASE_URL = os.environ.get("BASE_URL")
BASE_URL = os.environ.get("ORCHARD_TEST_URL")
if not BASE_URL:
print("ERROR: BASE_URL not set")
print("ERROR: ORCHARD_TEST_URL not set")
sys.exit(1)
print(f"Running integration tests against {BASE_URL}")
print(f"Running smoke tests against {BASE_URL}")
client = httpx.Client(base_url=BASE_URL, timeout=30.0)
errors = []
@@ -86,49 +192,138 @@ kics:
print(f" FAIL: {e}")
sys.exit(1)
else:
print("SUCCESS: All integration tests passed!")
print("SUCCESS: All smoke tests passed!")
sys.exit(0)
PYTEST_SCRIPT
# Integration tests for stage deployment
integration_test_stage:
<<: *integration_test_template
needs: [deploy_stage]
variables:
BASE_URL: https://orchard-stage.common.global.bsf.tools
# Integration tests for stage deployment (full suite)
# Reset stage template - shared by pre and post test reset jobs
# Calls the /api/v1/admin/factory-reset endpoint which handles DB and S3 cleanup
.reset_stage_template: &reset_stage_template
stage: deploy
image: deps.global.bsf.tools/docker/python:3.12-slim
timeout: 5m
retry: 1 # Retry once on transient failures
before_script:
- pip install --index-url "$PIP_INDEX_URL" httpx
script:
- |
python - <<'RESET_SCRIPT'
import httpx
import sys
import os
import time
BASE_URL = os.environ.get("STAGE_URL", "")
ADMIN_USER = "admin"
ADMIN_PASS = "changeme123" # Default admin password
MAX_RETRIES = 3
RETRY_DELAY = 5 # seconds
if not BASE_URL:
print("ERROR: STAGE_URL environment variable not set")
sys.exit(1)
print(f"=== Resetting stage environment at {BASE_URL} ===")
def do_reset():
with httpx.Client(base_url=BASE_URL, timeout=120.0) as client:
# Login as admin
print("Logging in as admin...")
login_response = client.post(
"/api/v1/auth/login",
json={"username": ADMIN_USER, "password": ADMIN_PASS},
)
if login_response.status_code != 200:
raise Exception(f"Login failed: {login_response.status_code} - {login_response.text}")
print("Login successful")
# Call factory reset endpoint
print("Calling factory reset endpoint...")
reset_response = client.post(
"/api/v1/admin/factory-reset",
headers={"X-Confirm-Reset": "yes-delete-all-data"},
)
if reset_response.status_code == 200:
result = reset_response.json()
print("Factory reset successful!")
print(f" Database tables dropped: {result['results']['database_tables_dropped']}")
print(f" S3 objects deleted: {result['results']['s3_objects_deleted']}")
print(f" Database reinitialized: {result['results']['database_reinitialized']}")
print(f" Seeded: {result['results']['seeded']}")
return True
else:
raise Exception(f"Factory reset failed: {reset_response.status_code} - {reset_response.text}")
# Retry loop
for attempt in range(1, MAX_RETRIES + 1):
try:
print(f"Attempt {attempt}/{MAX_RETRIES}")
if do_reset():
sys.exit(0)
except Exception as e:
print(f"Attempt {attempt} failed: {e}")
if attempt < MAX_RETRIES:
print(f"Retrying in {RETRY_DELAY} seconds...")
time.sleep(RETRY_DELAY)
else:
print("All retry attempts failed")
sys.exit(1)
RESET_SCRIPT
rules:
- if: '$CI_COMMIT_BRANCH == "main"'
when: on_success
# Integration tests for feature deployment
# Reset stage BEFORE integration tests (ensure known state)
reset_stage_pre:
<<: *reset_stage_template
needs: [deploy_stage]
# Integration tests for stage deployment (full suite)
integration_test_stage:
<<: *integration_test_template
needs: [reset_stage_pre]
variables:
ORCHARD_TEST_URL: $STAGE_URL
rules:
- if: '$CI_COMMIT_BRANCH == "main"'
when: on_success
# Reset stage AFTER integration tests (clean slate for next run)
reset_stage:
<<: *reset_stage_template
needs: [integration_test_stage]
allow_failure: true # Don't fail pipeline if reset has issues
# Integration tests for feature deployment (full suite)
integration_test_feature:
<<: *integration_test_template
needs: [deploy_feature]
variables:
BASE_URL: https://orchard-$CI_COMMIT_REF_SLUG.common.global.bsf.tools
ORCHARD_TEST_URL: https://orchard-$CI_COMMIT_REF_SLUG.common.global.bsf.tools
rules:
- if: '$CI_COMMIT_BRANCH && $CI_COMMIT_BRANCH != "main"'
when: on_success
# Run Python backend tests
python_tests:
# Run Python backend unit tests
python_unit_tests:
stage: test
needs: [] # Run in parallel with build
image: deps.global.bsf.tools/docker/python:3.12-slim
timeout: 15m
variables:
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.pip-cache"
interruptible: true # Cancel if new pipeline starts
cache:
key: pip-$CI_COMMIT_REF_SLUG
paths:
- .pip-cache/
policy: pull-push
before_script:
- pip install -r backend/requirements.txt
- pip install pytest pytest-asyncio pytest-cov httpx
- pip install --index-url "$PIP_INDEX_URL" -r backend/requirements.txt
- pip install --index-url "$PIP_INDEX_URL" pytest pytest-asyncio pytest-cov httpx
script:
- cd backend
# Only run unit tests - integration tests require Docker Compose services
# Run unit tests (integration tests run post-deployment against live environment)
- python -m pytest tests/unit/ -v --cov=app --cov-report=term --cov-report=xml:coverage.xml --cov-report=html:coverage_html --junitxml=pytest-report.xml
artifacts:
when: always
@@ -143,6 +338,10 @@ python_tests:
coverage_format: cobertura
path: backend/coverage.xml
coverage: '/TOTAL.*\s+(\d+%)/'
rules:
- if: '$CI_COMMIT_TAG'
when: never
- when: on_success
# Run frontend tests
frontend_tests:
@@ -150,6 +349,7 @@ frontend_tests:
needs: [] # Run in parallel with build
image: deps.global.bsf.tools/docker/node:20-alpine
timeout: 15m
interruptible: true # Cancel if new pipeline starts
cache:
key: npm-$CI_COMMIT_REF_SLUG
paths:
@@ -171,11 +371,15 @@ frontend_tests:
coverage_format: cobertura
path: frontend/coverage/cobertura-coverage.xml
coverage: '/All files[^|]*\|[^|]*\s+([\d\.]+)/'
rules:
- if: '$CI_COMMIT_TAG'
when: never
- when: on_success
# Shared deploy configuration
.deploy_template: &deploy_template
stage: deploy
needs: [build_image, kics, hadolint, python_tests, frontend_tests, secrets]
needs: [build_image, test_image, kics, hadolint, python_unit_tests, frontend_tests, secrets, app_deps_scan, cve_scan, cve_sbom_analysis, app_sbom_analysis]
image: deps.global.bsf.tools/registry-1.docker.io/alpine/k8s:1.29.12
.helm_setup: &helm_setup
@@ -184,47 +388,21 @@ frontend_tests:
# OCI-based charts from internal registry - no repo add needed
- helm dependency update
# Simplified deployment verification - just health check
# Full API/frontend checks are done by integration tests post-deployment
.verify_deployment: &verify_deployment |
echo "=== Waiting for health endpoint (certs may take a few minutes) ==="
for i in $(seq 1 30); do
if curl -sf --max-time 10 "$BASE_URL/health" > /dev/null 2>&1; then
echo "Health check passed!"
break
echo "Deployment URL: $BASE_URL"
exit 0
fi
echo "Attempt $i/30 - waiting 10s..."
sleep 10
done
# Verify health endpoint
echo ""
echo "=== Health Check ==="
curl -sf "$BASE_URL/health" || { echo "Health check failed"; exit 1; }
echo ""
# Verify API is responding
echo ""
echo "=== API Check (GET /api/v1/projects) ==="
HTTP_CODE=$(curl -sf -o /dev/null -w "%{http_code}" "$BASE_URL/api/v1/projects")
if [ "$HTTP_CODE" = "200" ]; then
echo "API responding: HTTP $HTTP_CODE"
else
echo "API check failed: HTTP $HTTP_CODE"
exit 1
fi
# Verify frontend is served
echo ""
echo "=== Frontend Check ==="
if curl -sf "$BASE_URL/" | grep -q "</html>"; then
echo "Frontend is being served"
else
echo "Frontend check failed"
exit 1
fi
echo ""
echo "=== All checks passed! ==="
echo "Deployment URL: $BASE_URL"
echo "Health check failed after 30 attempts"
exit 1
# Deploy to stage (main branch)
deploy_stage:
@@ -232,7 +410,7 @@ deploy_stage:
variables:
NAMESPACE: orch-stage-namespace
VALUES_FILE: helm/orchard/values-stage.yaml
BASE_URL: https://orchard-stage.common.global.bsf.tools
BASE_URL: $STAGE_URL
before_script:
- kubectl config use-context esv/bsf/bsf-integration/orchard/orchard-mvp:orchard-stage
- *helm_setup
@@ -246,17 +424,17 @@ deploy_stage:
--set image.tag=git.linux-amd64-$CI_COMMIT_SHA \
--wait \
--atomic \
--timeout 5m
- kubectl rollout status deployment/orchard-stage-server -n $NAMESPACE --timeout=5m
--timeout 10m
- kubectl rollout status deployment/orchard-stage-server -n $NAMESPACE --timeout=10m
- *verify_deployment
environment:
name: stage
url: https://orchard-stage.common.global.bsf.tools
url: $STAGE_URL
kubernetes:
agent: esv/bsf/bsf-integration/orchard/orchard-mvp:orchard-stage
rules:
- if: '$CI_COMMIT_BRANCH == "main"'
when: always
when: on_success
# Deploy feature branch to dev namespace
deploy_feature:
@@ -282,8 +460,8 @@ deploy_feature:
--set minioIngress.tls.secretName=minio-$CI_COMMIT_REF_SLUG-tls \
--wait \
--atomic \
--timeout 5m
- kubectl rollout status deployment/orchard-$CI_COMMIT_REF_SLUG-server -n $NAMESPACE --timeout=5m
--timeout 10m
- kubectl rollout status deployment/orchard-$CI_COMMIT_REF_SLUG-server -n $NAMESPACE --timeout=10m
- export BASE_URL="https://orchard-$CI_COMMIT_REF_SLUG.common.global.bsf.tools"
- *verify_deployment
environment:
@@ -295,12 +473,14 @@ deploy_feature:
agent: esv/bsf/bsf-integration/orchard/orchard-mvp:orchard
rules:
- if: '$CI_COMMIT_BRANCH && $CI_COMMIT_BRANCH != "main"'
when: always
when: on_success
# Cleanup feature branch deployment
# Cleanup feature branch deployment (standalone - doesn't need deploy dependencies)
cleanup_feature:
<<: *deploy_template
stage: deploy
needs: []
image: deps.global.bsf.tools/registry-1.docker.io/alpine/k8s:1.29.12
timeout: 5m
variables:
NAMESPACE: orch-dev-namespace
GIT_STRATEGY: none # No source needed, branch may be deleted
@@ -318,3 +498,50 @@ cleanup_feature:
- if: '$CI_COMMIT_BRANCH && $CI_COMMIT_BRANCH != "main"'
when: manual
allow_failure: true
# Deploy to production (version tags only)
deploy_prod:
stage: deploy
# For tag pipelines, no other jobs run - image was already built when commit was on main
needs: []
image: deps.global.bsf.tools/registry-1.docker.io/alpine/k8s:1.29.12
variables:
NAMESPACE: orch-namespace
VALUES_FILE: helm/orchard/values-prod.yaml
BASE_URL: $PROD_URL
before_script:
- kubectl config use-context esv/bsf/bsf-integration/orchard/orchard-mvp:orchard-prod
- *helm_setup
script:
- echo "Deploying to PRODUCTION - version $CI_COMMIT_TAG"
- cd $CI_PROJECT_DIR
- |
helm upgrade --install orchard-prod ./helm/orchard \
--namespace $NAMESPACE \
-f $VALUES_FILE \
--set image.tag=git.linux-amd64-$CI_COMMIT_SHA \
--wait \
--atomic \
--timeout 10m
- kubectl rollout status deployment/orchard-prod-server -n $NAMESPACE --timeout=10m
- *verify_deployment
environment:
name: production
url: $PROD_URL
kubernetes:
agent: esv/bsf/bsf-integration/orchard/orchard-mvp:orchard-prod
rules:
# Only run on semantic version tags (v1.0.0, v1.2.3, etc.)
- if: '$CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/'
when: on_success
allow_failure: false
# Smoke tests for production deployment (read-only, no test data creation)
smoke_test_prod:
<<: *smoke_test_template
needs: [deploy_prod]
variables:
ORCHARD_TEST_URL: $PROD_URL
rules:
- if: '$CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/'
when: on_success

8
.gitleaks.toml Normal file
View File

@@ -0,0 +1,8 @@
# Gitleaks configuration
# https://github.com/gitleaks/gitleaks#configuration
[allowlist]
# Test files that contain variable names matching secret patterns (e.g., s3_key)
paths = [
'''backend/tests/.*\.py''',
]

View File

@@ -4,6 +4,7 @@
# False positive: s3_key is an attribute name in test assertions, not a secret
# These are historical commits - files have since been deleted or updated with inline comments
7e68baed0886a3c928644cd01aa3b39f92d4f976:backend/tests/test_duplicate_detection.py:generic-api-key:154
81458b3bcb5ace97109ba4c16f4afa6e55b1b8bd:backend/tests/test_duplicate_detection.py:generic-api-key:154
2f1891cf0126ec0e7d4c789d872a2cb2dd3a1745:backend/tests/unit/test_storage.py:generic-api-key:381
10d36947948de796f0bacea3827f4531529c405d:backend/tests/unit/test_storage.py:generic-api-key:381
bccbc71c13570d14b8b26a11335c45f102fe3072:backend/tests/unit/test_storage.py:generic-api-key:381
@@ -11,3 +12,8 @@ bccbc71c13570d14b8b26a11335c45f102fe3072:backend/tests/unit/test_storage.py:gene
90bb2a3a393d2361dc3136ee8d761debb0726d8a:backend/tests/unit/test_storage.py:generic-api-key:381
37666e41a72d2a4f34447c0d1a8728e1d7271d24:backend/tests/unit/test_storage.py:generic-api-key:381
0cc4f253621a9601c5193f6ae1e7ae33f0e7fc9b:backend/tests/unit/test_storage.py:generic-api-key:381
35fda65d381acc5ab59bc592ee3013f75906c197:backend/tests/unit/test_storage.py:generic-api-key:381
08dce6cbb836b687002751fed4159bfc2da61f8b:backend/tests/unit/test_storage.py:generic-api-key:381
617bcbe89cff9a009d77e4f1f1864efed1820e63:backend/tests/unit/test_storage.py:generic-api-key:381
1cbd33544388e0fe6db752fa8886fab33cf9ce7c:backend/tests/unit/test_storage.py:generic-api-key:381
7cfad28f678f5a5b8b927d694a17b9ba446b7138:backend/tests/unit/test_storage.py:generic-api-key:381

View File

@@ -7,6 +7,107 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Added
- Added Project Settings page accessible to project admins (#65)
- General settings section for editing description and visibility
- Access Management section (moved from project page)
- Danger Zone section with inline delete confirmation requiring project name
- Settings button (gear icon) on project page header for admins
- Added artifact dependency management system (#76, #77, #78, #79, #80, #81)
- `artifact_dependencies` table with version/tag constraints and check constraints
- `ArtifactDependency` SQLAlchemy model with indexes for fast lookups
- Ensure file parsing (`orchard.ensure` YAML format) during artifact upload
- Circular dependency detection at upload time (rejected with 400)
- Dependency conflict detection at resolution time (409 with conflict details)
- Added dependency API endpoints (#78, #79):
- `GET /api/v1/artifact/{artifact_id}/dependencies` - Get dependencies by artifact ID
- `GET /api/v1/project/{project}/{package}/+/{ref}/dependencies` - Get dependencies by ref
- `GET /api/v1/project/{project}/{package}/reverse-dependencies` - Get reverse dependencies (paginated)
- `GET /api/v1/project/{project}/{package}/+/{ref}/resolve` - Resolve full dependency tree
- Added dependency resolution with topological sorting (#79)
- Returns flat list of all artifacts needed in dependency order
- Includes download URLs, sizes, and version info for each artifact
- Added frontend dependency visualization (#84, #85, #86):
- Dependencies section on package page showing direct dependencies for selected tag
- Tag/version selector to switch between artifacts
- "Used By" section showing reverse dependencies with pagination
- Interactive dependency graph modal with:
- Tree visualization with collapsible nodes
- Zoom (mouse wheel + buttons) and pan (click-drag)
- Click to navigate to package
- Hover tooltip with package details
- Error display for circular dependencies and conflicts
- Added migration `008_artifact_dependencies.sql` for dependency schema
- Added `dependencies.py` module with parsing, validation, and resolution logic
- Added comprehensive integration tests for all dependency features
### Changed
- Added pre-test stage reset to ensure known environment state before integration tests (#54)
- Upload endpoint now accepts optional `ensure` file parameter for declaring dependencies
- Updated upload API documentation with ensure file format and examples
## [0.5.1] - 2026-01-23
### Changed
- Simplified tag pipeline to only run deploy and smoke tests (image already built on main) (#54)
### Fixed
- Fixed production CI deployment namespace to use correct `orch-namespace` (#54)
- Added gitleaks config to allowlist test files from secret scanning (#54)
## [0.5.0] - 2026-01-23
### Added
- Added factory reset endpoint `POST /api/v1/admin/factory-reset` for test environment cleanup (#54)
- Requires admin authentication and `X-Confirm-Reset: yes-delete-all-data` header
- Drops all database tables, clears S3 bucket, reinitializes schema, re-seeds default data
- CI pipeline automatically calls this after integration tests on stage
- Added `delete_all()` method to storage backend for bulk S3 object deletion (#54)
- Added AWS Secrets Manager CSI driver support for database credentials (#54)
- Added SecretProviderClass template for Secrets Manager integration (#54)
- Added IRSA service account annotations for prod and stage environments (#54)
- Added comprehensive upload/download tests for size boundaries (1B to 1GB) (#38)
- Added concurrent upload/download tests (2, 5, 10 parallel operations) (#38)
- Added data integrity tests (binary, text, unicode, compressed content) (#38)
- Added chunk boundary tests for edge cases (#38)
- Added `@pytest.mark.large` and `@pytest.mark.concurrent` test markers (#38)
- Added `generate_content()` and `generate_content_with_hash()` test helpers (#38)
- Added `sized_content` fixture for generating test content of specific sizes (#38)
- Added upload API tests: upload without tag, artifact creation verification, S3 object creation (#38)
- Added download API tests: tag: prefix resolution, 404 for nonexistent project/package/artifact (#38)
- Added download header tests: Content-Type, Content-Length, Content-Disposition, ETag, X-Checksum-SHA256 (#38)
- Added error handling tests: timeout behavior, checksum validation, resource cleanup, graceful error responses (#38)
- Added version API tests: version creation, auto-detection, listing, download by version prefix (#38)
- Added integrity verification tests: round-trip hash verification, client-side verification workflow, size variants (1KB-10MB) (#40)
- Added consistency check endpoint tests with response format validation (#40)
- Added corruption detection tests: bit flip, truncation, appended content, size mismatch, missing S3 objects (#40)
- Added Digest header tests (RFC 3230) and verification mode tests (#40)
- Added integrity verification documentation (`docs/integrity-verification.md`) (#40)
- Added conditional request support for downloads (If-None-Match, If-Modified-Since) returning 304 Not Modified (#42)
- Added caching headers to downloads: Cache-Control (immutable), Last-Modified (#42)
- Added 416 Range Not Satisfiable response for invalid range requests (#42)
- Added download completion logging with bytes transferred and throughput (#42)
- Added client disconnect handling during streaming downloads (#42)
- Added streaming download tests: range requests, conditional requests, caching headers, download resume (#42)
- Added upload duration and throughput metrics (`duration_ms`, `throughput_mbps`) to upload response (#43)
- Added upload progress logging for large files (hash computation and multipart upload phases) (#43)
- Added client disconnect handling during uploads with proper cleanup (#43)
- Added upload progress tracking endpoint `GET /upload/{upload_id}/progress` for resumable uploads (#43)
- Added large file upload tests (10MB, 100MB, 1GB) with multipart upload verification (#43)
- Added upload cancellation and timeout handling tests (#43)
- Added comprehensive API documentation for upload endpoints with curl, Python, and JavaScript examples (#43)
- Added `package_versions` table for immutable version tracking separate from mutable tags (#56)
- Versions are set at upload time via explicit `version` parameter or auto-detected from filename/metadata
- Version detection priority: explicit parameter > package metadata > filename pattern
- Versions are immutable once created (unlike tags which can be moved)
- Added version API endpoints (#56):
- `GET /api/v1/project/{project}/{package}/versions` - List all versions for a package
- `GET /api/v1/project/{project}/{package}/versions/{version}` - Get specific version details
- `DELETE /api/v1/project/{project}/{package}/versions/{version}` - Delete a version (admin only)
- Added version support to upload endpoint via `version` form parameter (#56)
- Added `version:X.Y.Z` prefix for explicit version resolution in download refs (#56)
- Added version field to tag responses (shows which version the artifact has, if any) (#56)
- Added migration `007_package_versions.sql` with ref_count triggers and data migration from semver tags (#56)
- Added production deployment job triggered by semantic version tags (v1.0.0) with manual approval gate (#63)
- Added production Helm values file with persistence enabled (20Gi PostgreSQL, 100Gi MinIO) (#63)
- Added integration tests for production deployment (#63)
- Added GitLab CI pipeline for feature branch deployments to dev namespace (#51)
- Added `deploy_feature` job with dynamic hostnames and unique release names (#51)
- Added `cleanup_feature` job with `on_stop` for automatic cleanup on merge (#51)
@@ -16,6 +117,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Added internal proxy configuration for npm, pip, helm, and apt (#51)
### Changed
- Configured stage and prod to use AWS RDS instead of PostgreSQL subchart (#54)
- Configured stage and prod to use AWS S3 instead of MinIO subchart (#54)
- Changed prod deployment from manual to automatic on version tags (#54)
- Updated S3 client to support IRSA credentials when no explicit keys provided (#54)
- Changed prod image pullPolicy to Always (#54)
- Added proxy-body-size annotation to prod ingress for large uploads (#54)
- CI integration tests now run full pytest suite (~350 tests) against deployed environment instead of 3 smoke tests
- CI production deployment uses lightweight smoke tests only (no test data creation in prod)
- CI pipeline improvements: shared pip cache, `interruptible` flag on test jobs, retry on integration tests
- Simplified deploy verification to health check only (full checks done by integration tests)
- Extracted environment URLs to global variables for maintainability
- Made `cleanup_feature` job standalone (no longer inherits deploy template dependencies)
- Renamed `integration_test_prod` to `smoke_test_prod` for clarity
- Updated download ref resolution to check versions before tags (version → tag → artifact ID) (#56)
- Deploy jobs now require all security scans to pass before deployment (added test_image, app_deps_scan, cve_scan, cve_sbom_analysis, app_sbom_analysis to dependencies) (#63)
- Increased deploy job timeout from 5m to 10m (#63)
- Added `--atomic` flag to Helm deployments for automatic rollback on failure
- Adjusted dark mode color palette to use lighter background tones for better readability and reduced eye strain (#52)
- Replaced project card grid with sortable data table on Home page for better handling of large project lists
@@ -26,6 +143,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Improved pod naming: Orchard pods now named `orchard-{env}-server-*` for clarity (#51)
### Fixed
- Fixed factory reset not creating default admin user after reset (#60)
- Admin user was only created at server startup, not after factory reset
- CI reset job would fail to login because admin user didn't exist
- Improved reset_stage CI job reliability (#60)
- Added application-level retry logic (3 attempts with 5s delay)
- Added job-level retry for transient failures
- Fixed httpx client to use proper context manager
- Increased timeout to 120s for reset operations
- Fixed CI integration test rate limiting: added configurable `ORCHARD_LOGIN_RATE_LIMIT` env var, relaxed to 1000/minute for dev/stage
- Fixed duplicate `TestSecurityEdgeCases` class definition in test_auth_api.py
- Fixed integration tests auth: session-scoped client, configurable credentials via env vars, fail-fast on auth errors
- Fixed 413 Request Entity Too Large errors on uploads by adding `proxy-body-size: "0"` nginx annotation to Orchard ingress
- Fixed CI tests that require direct S3 access: added `@pytest.mark.requires_direct_s3` marker and excluded from CI
- Fixed ref_count triggers not being created: added auto-migration for tags ref_count trigger functions
- Fixed Content-Disposition header encoding for non-ASCII filenames using RFC 5987 (#38)
- Fixed deploy jobs running even when tests or security scans fail (changed rules from `when: always` to `when: on_success`) (#63)
- Fixed python_tests job not using internal PyPI proxy (#63)
- Fixed `cleanup_feature` job failing when branch is deleted (`GIT_STRATEGY: none`) (#51)
- Fixed gitleaks false positives with fingerprints for historical commits (#51)
- Fixed integration tests running when deploy fails (`when: on_success`) (#51)
@@ -33,9 +167,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Fixed deploy jobs running when secrets scan fails (added `secrets` to deploy dependencies)
- Fixed dev environment memory requests to equal limits per cluster Kyverno policy
- Fixed init containers missing resource limits (Kyverno policy compliance)
- Fixed Python SyntaxWarning for invalid escape sequence in database migration regex pattern
### Removed
- Removed unused `store_streaming()` method from storage.py (#51)
- Disabled PostgreSQL subchart for stage and prod environments (#54)
- Disabled MinIO subchart for stage and prod environments (#54)
## [0.4.0] - 2026-01-12
### Added

View File

@@ -22,6 +22,7 @@ Orchard is a centralized binary artifact storage system that provides content-ad
- **Package** - Named collection within a project
- **Artifact** - Specific content instance identified by SHA256
- **Tags** - Alias system for referencing artifacts by human-readable names (e.g., `v1.0.0`, `latest`, `stable`)
- **Versions** - Immutable version records set at upload time (explicit or auto-detected from filename/metadata), separate from mutable tags
- **Package Formats & Platforms** - Packages can be tagged with format (npm, pypi, docker, deb, rpm, etc.) and platform (linux, darwin, windows, etc.)
- **Rich Package Metadata** - Package listings include aggregated stats (tag count, artifact count, total size, latest tag)
- **S3-Compatible Backend** - Uses MinIO (or any S3-compatible storage) for artifact storage
@@ -73,6 +74,9 @@ Orchard is a centralized binary artifact storage system that provides content-ad
| `POST` | `/api/v1/project/:project/:package/tags` | Create a tag |
| `GET` | `/api/v1/project/:project/:package/tags/:tag_name` | Get single tag with artifact metadata |
| `GET` | `/api/v1/project/:project/:package/tags/:tag_name/history` | Get tag change history |
| `GET` | `/api/v1/project/:project/:package/versions` | List all versions for a package |
| `GET` | `/api/v1/project/:project/:package/versions/:version` | Get specific version details |
| `DELETE` | `/api/v1/project/:project/:package/versions/:version` | Delete a version (admin only) |
| `GET` | `/api/v1/project/:project/:package/artifacts` | List artifacts in package (with filtering) |
| `GET` | `/api/v1/project/:project/:package/consumers` | List consumers of a package |
| `GET` | `/api/v1/artifact/:id` | Get artifact metadata with referencing tags |
@@ -93,12 +97,14 @@ For large files, use the resumable upload API:
When downloading artifacts, the `:ref` parameter supports multiple formats:
- `latest` - Tag name directly
- `v1.0.0` - Version tag
- `latest` - Implicit lookup (checks version first, then tag, then artifact ID)
- `v1.0.0` - Implicit lookup (version takes precedence over tag with same name)
- `version:1.0.0` - Explicit version reference
- `tag:stable` - Explicit tag reference
- `version:2024.1` - Version reference
- `artifact:a3f5d8e12b4c6789...` - Direct SHA256 hash reference
**Resolution order for implicit refs:** version → tag → artifact ID
## Quick Start
### Prerequisites
@@ -230,9 +236,16 @@ curl "http://localhost:8080/api/v1/project/my-project/packages/releases?include_
### Upload an Artifact
```bash
# Upload with tag only (version auto-detected from filename)
curl -X POST http://localhost:8080/api/v1/project/my-project/releases/upload \
-F "file=@./build/app-v1.0.0.tar.gz" \
-F "tag=v1.0.0"
-F "tag=latest"
# Upload with explicit version and tag
curl -X POST http://localhost:8080/api/v1/project/my-project/releases/upload \
-F "file=@./build/app-v1.0.0.tar.gz" \
-F "tag=latest" \
-F "version=1.0.0"
```
Response:
@@ -242,7 +255,9 @@ Response:
"size": 1048576,
"project": "my-project",
"package": "releases",
"tag": "v1.0.0",
"tag": "latest",
"version": "1.0.0",
"version_source": "explicit",
"format_metadata": {
"format": "tarball",
"package_name": "app",
@@ -400,6 +415,38 @@ curl http://localhost:8080/api/v1/project/my-project/releases/tags/latest/histor
Returns list of artifact changes for the tag (most recent first).
### List Versions
```bash
# Basic listing
curl http://localhost:8080/api/v1/project/my-project/releases/versions
# With pagination and sorting
curl "http://localhost:8080/api/v1/project/my-project/releases/versions?sort=version&order=desc"
```
Response includes tags pointing to each version's artifact:
```json
{
"items": [
{
"id": "uuid",
"package_id": "uuid",
"version": "1.0.0",
"version_source": "explicit",
"artifact_id": "a3f5d8e...",
"size": 1048576,
"content_type": "application/gzip",
"original_name": "app-v1.0.0.tar.gz",
"created_at": "2025-01-01T00:00:00Z",
"created_by": "user",
"tags": ["latest", "stable"]
}
],
"pagination": {"page": 1, "limit": 20, "total": 1, "total_pages": 1}
}
```
### List Artifacts in Package
```bash
@@ -614,7 +661,8 @@ See `helm/orchard/values.yaml` for all configuration options.
- **projects** - Top-level organizational containers
- **packages** - Collections within projects
- **artifacts** - Content-addressable artifacts (SHA256)
- **tags** - Aliases pointing to artifacts
- **tags** - Mutable aliases pointing to artifacts
- **package_versions** - Immutable version records (set at upload time)
- **tag_history** - Audit trail for tag changes
- **uploads** - Upload event records
- **consumers** - Dependency tracking

View File

@@ -151,6 +151,140 @@ def _run_migrations():
END IF;
END $$;
""",
# Add package_versions indexes and triggers (007_package_versions.sql)
"""
DO $$
BEGIN
-- Create indexes for package_versions if table exists
IF EXISTS (SELECT 1 FROM information_schema.tables WHERE table_name = 'package_versions') THEN
-- Indexes for common queries
IF NOT EXISTS (SELECT 1 FROM pg_indexes WHERE indexname = 'idx_package_versions_package_id') THEN
CREATE INDEX idx_package_versions_package_id ON package_versions(package_id);
END IF;
IF NOT EXISTS (SELECT 1 FROM pg_indexes WHERE indexname = 'idx_package_versions_artifact_id') THEN
CREATE INDEX idx_package_versions_artifact_id ON package_versions(artifact_id);
END IF;
IF NOT EXISTS (SELECT 1 FROM pg_indexes WHERE indexname = 'idx_package_versions_package_version') THEN
CREATE INDEX idx_package_versions_package_version ON package_versions(package_id, version);
END IF;
END IF;
END $$;
""",
# Create ref_count trigger functions for tags (ensures triggers exist even if initial migration wasn't run)
"""
CREATE OR REPLACE FUNCTION increment_artifact_ref_count()
RETURNS TRIGGER AS $$
BEGIN
UPDATE artifacts SET ref_count = ref_count + 1 WHERE id = NEW.artifact_id;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
""",
"""
CREATE OR REPLACE FUNCTION decrement_artifact_ref_count()
RETURNS TRIGGER AS $$
BEGIN
UPDATE artifacts SET ref_count = ref_count - 1 WHERE id = OLD.artifact_id;
RETURN OLD;
END;
$$ LANGUAGE plpgsql;
""",
"""
CREATE OR REPLACE FUNCTION update_artifact_ref_count()
RETURNS TRIGGER AS $$
BEGIN
IF OLD.artifact_id != NEW.artifact_id THEN
UPDATE artifacts SET ref_count = ref_count - 1 WHERE id = OLD.artifact_id;
UPDATE artifacts SET ref_count = ref_count + 1 WHERE id = NEW.artifact_id;
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
""",
# Create triggers for tags ref_count management
"""
DO $$
BEGIN
-- Drop and recreate triggers to ensure they're current
DROP TRIGGER IF EXISTS tags_ref_count_insert_trigger ON tags;
CREATE TRIGGER tags_ref_count_insert_trigger
AFTER INSERT ON tags
FOR EACH ROW
EXECUTE FUNCTION increment_artifact_ref_count();
DROP TRIGGER IF EXISTS tags_ref_count_delete_trigger ON tags;
CREATE TRIGGER tags_ref_count_delete_trigger
AFTER DELETE ON tags
FOR EACH ROW
EXECUTE FUNCTION decrement_artifact_ref_count();
DROP TRIGGER IF EXISTS tags_ref_count_update_trigger ON tags;
CREATE TRIGGER tags_ref_count_update_trigger
AFTER UPDATE ON tags
FOR EACH ROW
WHEN (OLD.artifact_id IS DISTINCT FROM NEW.artifact_id)
EXECUTE FUNCTION update_artifact_ref_count();
END $$;
""",
# Create ref_count trigger functions for package_versions
"""
CREATE OR REPLACE FUNCTION increment_version_ref_count()
RETURNS TRIGGER AS $$
BEGIN
UPDATE artifacts SET ref_count = ref_count + 1 WHERE id = NEW.artifact_id;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
""",
"""
CREATE OR REPLACE FUNCTION decrement_version_ref_count()
RETURNS TRIGGER AS $$
BEGIN
UPDATE artifacts SET ref_count = ref_count - 1 WHERE id = OLD.artifact_id;
RETURN OLD;
END;
$$ LANGUAGE plpgsql;
""",
# Create triggers for package_versions ref_count
"""
DO $$
BEGIN
IF EXISTS (SELECT 1 FROM information_schema.tables WHERE table_name = 'package_versions') THEN
-- Drop and recreate triggers to ensure they're current
DROP TRIGGER IF EXISTS package_versions_ref_count_insert ON package_versions;
CREATE TRIGGER package_versions_ref_count_insert
AFTER INSERT ON package_versions
FOR EACH ROW
EXECUTE FUNCTION increment_version_ref_count();
DROP TRIGGER IF EXISTS package_versions_ref_count_delete ON package_versions;
CREATE TRIGGER package_versions_ref_count_delete
AFTER DELETE ON package_versions
FOR EACH ROW
EXECUTE FUNCTION decrement_version_ref_count();
END IF;
END $$;
""",
# Migrate existing semver tags to package_versions
r"""
DO $$
BEGIN
IF EXISTS (SELECT 1 FROM information_schema.tables WHERE table_name = 'package_versions') THEN
-- Migrate tags that look like versions (v1.0.0, 1.2.3, 2.0.0-beta, etc.)
INSERT INTO package_versions (package_id, artifact_id, version, version_source, created_by, created_at)
SELECT
t.package_id,
t.artifact_id,
CASE WHEN t.name LIKE 'v%' THEN substring(t.name from 2) ELSE t.name END,
'migrated_from_tag',
t.created_by,
t.created_at
FROM tags t
WHERE t.name ~ '^v?[0-9]+\.[0-9]+(\.[0-9]+)?([-.][a-zA-Z0-9]+)?$'
ON CONFLICT (package_id, version) DO NOTHING;
END IF;
END $$;
""",
]
with engine.connect() as conn:

723
backend/app/dependencies.py Normal file
View File

@@ -0,0 +1,723 @@
"""
Dependency management module for artifact dependencies.
Handles:
- Parsing orchard.ensure files
- Storing dependencies in the database
- Querying dependencies and reverse dependencies
- Dependency resolution with topological sorting
- Circular dependency detection
- Conflict detection
"""
import yaml
from typing import List, Dict, Any, Optional, Set, Tuple
from sqlalchemy.orm import Session
from sqlalchemy import and_
from .models import (
Project,
Package,
Artifact,
Tag,
ArtifactDependency,
PackageVersion,
)
from .schemas import (
EnsureFileContent,
EnsureFileDependency,
DependencyResponse,
ArtifactDependenciesResponse,
DependentInfo,
ReverseDependenciesResponse,
ResolvedArtifact,
DependencyResolutionResponse,
DependencyConflict,
PaginationMeta,
)
class DependencyError(Exception):
"""Base exception for dependency errors."""
pass
class CircularDependencyError(DependencyError):
"""Raised when a circular dependency is detected."""
def __init__(self, cycle: List[str]):
self.cycle = cycle
super().__init__(f"Circular dependency detected: {' -> '.join(cycle)}")
class DependencyConflictError(DependencyError):
"""Raised when conflicting dependency versions are detected."""
def __init__(self, conflicts: List[DependencyConflict]):
self.conflicts = conflicts
super().__init__(f"Dependency conflicts detected: {len(conflicts)} conflict(s)")
class DependencyNotFoundError(DependencyError):
"""Raised when a dependency cannot be resolved."""
def __init__(self, project: str, package: str, constraint: str):
self.project = project
self.package = package
self.constraint = constraint
super().__init__(f"Dependency not found: {project}/{package}@{constraint}")
class InvalidEnsureFileError(DependencyError):
"""Raised when the ensure file is invalid."""
pass
class DependencyDepthExceededError(DependencyError):
"""Raised when dependency resolution exceeds max depth."""
def __init__(self, max_depth: int):
self.max_depth = max_depth
super().__init__(f"Dependency resolution exceeded maximum depth of {max_depth}")
# Safety limits to prevent DoS attacks
MAX_DEPENDENCY_DEPTH = 50 # Maximum levels of nested dependencies
MAX_DEPENDENCIES_PER_ARTIFACT = 200 # Maximum direct dependencies per artifact
def parse_ensure_file(content: bytes) -> EnsureFileContent:
"""
Parse an orchard.ensure file.
Args:
content: Raw bytes of the ensure file
Returns:
Parsed EnsureFileContent
Raises:
InvalidEnsureFileError: If the file is invalid YAML or has wrong structure
"""
try:
data = yaml.safe_load(content.decode('utf-8'))
except yaml.YAMLError as e:
raise InvalidEnsureFileError(f"Invalid YAML: {e}")
except UnicodeDecodeError as e:
raise InvalidEnsureFileError(f"Invalid encoding: {e}")
if data is None:
return EnsureFileContent(dependencies=[])
if not isinstance(data, dict):
raise InvalidEnsureFileError("Ensure file must be a YAML dictionary")
dependencies = []
deps_data = data.get('dependencies', [])
if not isinstance(deps_data, list):
raise InvalidEnsureFileError("'dependencies' must be a list")
# Safety limit: prevent DoS through excessive dependencies
if len(deps_data) > MAX_DEPENDENCIES_PER_ARTIFACT:
raise InvalidEnsureFileError(
f"Too many dependencies: {len(deps_data)} exceeds maximum of {MAX_DEPENDENCIES_PER_ARTIFACT}"
)
for i, dep in enumerate(deps_data):
if not isinstance(dep, dict):
raise InvalidEnsureFileError(f"Dependency {i} must be a dictionary")
project = dep.get('project')
package = dep.get('package')
version = dep.get('version')
tag = dep.get('tag')
if not project:
raise InvalidEnsureFileError(f"Dependency {i} missing 'project'")
if not package:
raise InvalidEnsureFileError(f"Dependency {i} missing 'package'")
if not version and not tag:
raise InvalidEnsureFileError(
f"Dependency {i} must have either 'version' or 'tag'"
)
if version and tag:
raise InvalidEnsureFileError(
f"Dependency {i} cannot have both 'version' and 'tag'"
)
dependencies.append(EnsureFileDependency(
project=project,
package=package,
version=version,
tag=tag,
))
return EnsureFileContent(dependencies=dependencies)
def validate_dependencies(
db: Session,
dependencies: List[EnsureFileDependency],
) -> List[str]:
"""
Validate that all dependency projects exist.
Args:
db: Database session
dependencies: List of dependencies to validate
Returns:
List of error messages (empty if all valid)
"""
errors = []
for dep in dependencies:
project = db.query(Project).filter(Project.name == dep.project).first()
if not project:
errors.append(f"Project '{dep.project}' not found")
return errors
def store_dependencies(
db: Session,
artifact_id: str,
dependencies: List[EnsureFileDependency],
) -> List[ArtifactDependency]:
"""
Store dependencies for an artifact.
Args:
db: Database session
artifact_id: The artifact ID that has these dependencies
dependencies: List of dependencies to store
Returns:
List of created ArtifactDependency objects
"""
created = []
for dep in dependencies:
artifact_dep = ArtifactDependency(
artifact_id=artifact_id,
dependency_project=dep.project,
dependency_package=dep.package,
version_constraint=dep.version,
tag_constraint=dep.tag,
)
db.add(artifact_dep)
created.append(artifact_dep)
return created
def get_artifact_dependencies(
db: Session,
artifact_id: str,
) -> List[DependencyResponse]:
"""
Get all dependencies for an artifact.
Args:
db: Database session
artifact_id: The artifact ID
Returns:
List of DependencyResponse objects
"""
deps = db.query(ArtifactDependency).filter(
ArtifactDependency.artifact_id == artifact_id
).all()
return [DependencyResponse.from_orm_model(dep) for dep in deps]
def get_reverse_dependencies(
db: Session,
project_name: str,
package_name: str,
page: int = 1,
limit: int = 50,
) -> ReverseDependenciesResponse:
"""
Get all artifacts that depend on a given package.
Args:
db: Database session
project_name: Target project name
package_name: Target package name
page: Page number (1-indexed)
limit: Results per page
Returns:
ReverseDependenciesResponse with dependents and pagination
"""
# Query dependencies that point to this project/package
query = db.query(ArtifactDependency).filter(
ArtifactDependency.dependency_project == project_name,
ArtifactDependency.dependency_package == package_name,
)
total = query.count()
offset = (page - 1) * limit
deps = query.offset(offset).limit(limit).all()
dependents = []
for dep in deps:
# Get artifact info to find the project/package/version
artifact = db.query(Artifact).filter(Artifact.id == dep.artifact_id).first()
if not artifact:
continue
# Find which package this artifact belongs to via tags or versions
tag = db.query(Tag).filter(Tag.artifact_id == dep.artifact_id).first()
if tag:
pkg = db.query(Package).filter(Package.id == tag.package_id).first()
if pkg:
proj = db.query(Project).filter(Project.id == pkg.project_id).first()
if proj:
# Get version if available
version_record = db.query(PackageVersion).filter(
PackageVersion.artifact_id == dep.artifact_id,
PackageVersion.package_id == pkg.id,
).first()
dependents.append(DependentInfo(
artifact_id=dep.artifact_id,
project=proj.name,
package=pkg.name,
version=version_record.version if version_record else None,
constraint_type="version" if dep.version_constraint else "tag",
constraint_value=dep.version_constraint or dep.tag_constraint,
))
total_pages = (total + limit - 1) // limit
return ReverseDependenciesResponse(
project=project_name,
package=package_name,
dependents=dependents,
pagination=PaginationMeta(
page=page,
limit=limit,
total=total,
total_pages=total_pages,
has_more=page < total_pages,
),
)
def _resolve_dependency_to_artifact(
db: Session,
project_name: str,
package_name: str,
version: Optional[str],
tag: Optional[str],
) -> Optional[Tuple[str, str, int]]:
"""
Resolve a dependency constraint to an artifact ID.
Args:
db: Database session
project_name: Project name
package_name: Package name
version: Version constraint (exact)
tag: Tag constraint
Returns:
Tuple of (artifact_id, resolved_version_or_tag, size) or None if not found
"""
# Get project and package
project = db.query(Project).filter(Project.name == project_name).first()
if not project:
return None
package = db.query(Package).filter(
Package.project_id == project.id,
Package.name == package_name,
).first()
if not package:
return None
if version:
# Look up by version
pkg_version = db.query(PackageVersion).filter(
PackageVersion.package_id == package.id,
PackageVersion.version == version,
).first()
if pkg_version:
artifact = db.query(Artifact).filter(
Artifact.id == pkg_version.artifact_id
).first()
if artifact:
return (artifact.id, version, artifact.size)
# Also check if there's a tag with this exact name
tag_record = db.query(Tag).filter(
Tag.package_id == package.id,
Tag.name == version,
).first()
if tag_record:
artifact = db.query(Artifact).filter(
Artifact.id == tag_record.artifact_id
).first()
if artifact:
return (artifact.id, version, artifact.size)
if tag:
# Look up by tag
tag_record = db.query(Tag).filter(
Tag.package_id == package.id,
Tag.name == tag,
).first()
if tag_record:
artifact = db.query(Artifact).filter(
Artifact.id == tag_record.artifact_id
).first()
if artifact:
return (artifact.id, tag, artifact.size)
return None
def _detect_package_cycle(
db: Session,
project_name: str,
package_name: str,
target_project: str,
target_package: str,
visiting: Set[str],
visited: Set[str],
path: List[str],
) -> Optional[List[str]]:
"""
Detect cycles at the package level using DFS.
Args:
db: Database session
project_name: Current project being visited
package_name: Current package being visited
target_project: The project we're checking for cycles back to
target_package: The package we're checking for cycles back to
visiting: Set of package keys currently in the recursion stack
visited: Set of fully processed package keys
path: Current path for cycle reporting
Returns:
Cycle path if detected, None otherwise
"""
pkg_key = f"{project_name}/{package_name}"
# Check if we've reached the target package (cycle detected)
if project_name == target_project and package_name == target_package:
return path + [pkg_key]
if pkg_key in visiting:
# Unexpected internal cycle
return None
if pkg_key in visited:
return None
visiting.add(pkg_key)
path.append(pkg_key)
# Get the package and find any artifacts with dependencies
project = db.query(Project).filter(Project.name == project_name).first()
if project:
package = db.query(Package).filter(
Package.project_id == project.id,
Package.name == package_name,
).first()
if package:
# Find all artifacts in this package via tags
tags = db.query(Tag).filter(Tag.package_id == package.id).all()
artifact_ids = {t.artifact_id for t in tags}
# Get dependencies from all artifacts in this package
for artifact_id in artifact_ids:
deps = db.query(ArtifactDependency).filter(
ArtifactDependency.artifact_id == artifact_id
).all()
for dep in deps:
cycle = _detect_package_cycle(
db,
dep.dependency_project,
dep.dependency_package,
target_project,
target_package,
visiting,
visited,
path,
)
if cycle:
return cycle
path.pop()
visiting.remove(pkg_key)
visited.add(pkg_key)
return None
def check_circular_dependencies(
db: Session,
artifact_id: str,
new_dependencies: List[EnsureFileDependency],
project_name: Optional[str] = None,
package_name: Optional[str] = None,
) -> Optional[List[str]]:
"""
Check if adding the new dependencies would create a circular dependency.
Args:
db: Database session
artifact_id: The artifact that will have these dependencies
new_dependencies: Dependencies to be added
project_name: Project name (optional, will try to look up from tag if not provided)
package_name: Package name (optional, will try to look up from tag if not provided)
Returns:
Cycle path if detected, None otherwise
"""
# First, get the package info for this artifact to build path labels
if project_name and package_name:
current_path = f"{project_name}/{package_name}"
else:
# Try to look up from tag
artifact = db.query(Artifact).filter(Artifact.id == artifact_id).first()
if not artifact:
return None
# Find package for this artifact
tag = db.query(Tag).filter(Tag.artifact_id == artifact_id).first()
if not tag:
return None
package = db.query(Package).filter(Package.id == tag.package_id).first()
if not package:
return None
project = db.query(Project).filter(Project.id == package.project_id).first()
if not project:
return None
current_path = f"{project.name}/{package.name}"
# Extract target project and package from current_path
if "/" in current_path:
target_project, target_package = current_path.split("/", 1)
else:
return None
# For each new dependency, check if it would create a cycle back to our package
for dep in new_dependencies:
# Check if this dependency (transitively) depends on us at the package level
visiting: Set[str] = set()
visited: Set[str] = set()
path: List[str] = [current_path]
# Check from the dependency's package
cycle = _detect_package_cycle(
db,
dep.project,
dep.package,
target_project,
target_package,
visiting,
visited,
path,
)
if cycle:
return cycle
return None
def resolve_dependencies(
db: Session,
project_name: str,
package_name: str,
ref: str,
base_url: str,
) -> DependencyResolutionResponse:
"""
Resolve all dependencies for an artifact recursively.
Args:
db: Database session
project_name: Project name
package_name: Package name
ref: Tag or version reference
base_url: Base URL for download URLs
Returns:
DependencyResolutionResponse with all resolved artifacts
Raises:
DependencyNotFoundError: If a dependency cannot be resolved
CircularDependencyError: If circular dependencies are detected
DependencyConflictError: If conflicting versions are required
"""
# Resolve the initial artifact
project = db.query(Project).filter(Project.name == project_name).first()
if not project:
raise DependencyNotFoundError(project_name, package_name, ref)
package = db.query(Package).filter(
Package.project_id == project.id,
Package.name == package_name,
).first()
if not package:
raise DependencyNotFoundError(project_name, package_name, ref)
# Try to find artifact by tag or version
resolved = _resolve_dependency_to_artifact(
db, project_name, package_name, ref, ref
)
if not resolved:
raise DependencyNotFoundError(project_name, package_name, ref)
root_artifact_id, root_version, root_size = resolved
# Track resolved artifacts and their versions
resolved_artifacts: Dict[str, ResolvedArtifact] = {}
# Track version requirements for conflict detection
version_requirements: Dict[str, List[Dict[str, Any]]] = {} # pkg_key -> [(version, required_by)]
# Track visiting/visited for cycle detection
visiting: Set[str] = set()
visited: Set[str] = set()
# Resolution order (topological)
resolution_order: List[str] = []
def _resolve_recursive(
artifact_id: str,
proj_name: str,
pkg_name: str,
version_or_tag: str,
size: int,
required_by: Optional[str],
depth: int = 0,
):
"""Recursively resolve dependencies with cycle/conflict detection."""
# Safety limit: prevent DoS through deeply nested dependencies
if depth > MAX_DEPENDENCY_DEPTH:
raise DependencyDepthExceededError(MAX_DEPENDENCY_DEPTH)
pkg_key = f"{proj_name}/{pkg_name}"
# Cycle detection (at artifact level)
if artifact_id in visiting:
# Build cycle path
raise CircularDependencyError([pkg_key, pkg_key])
# Conflict detection - check if we've seen this package before with a different version
if pkg_key in version_requirements:
existing_versions = {r["version"] for r in version_requirements[pkg_key]}
if version_or_tag not in existing_versions:
# Conflict detected - same package, different version
requirements = version_requirements[pkg_key] + [
{"version": version_or_tag, "required_by": required_by}
]
raise DependencyConflictError([
DependencyConflict(
project=proj_name,
package=pkg_name,
requirements=[
{
"version": r["version"],
"required_by": [{"path": r["required_by"]}] if r["required_by"] else []
}
for r in requirements
],
)
])
# Same version already resolved - skip
if artifact_id in visited:
return
if artifact_id in visited:
return
visiting.add(artifact_id)
# Track version requirement
if pkg_key not in version_requirements:
version_requirements[pkg_key] = []
version_requirements[pkg_key].append({
"version": version_or_tag,
"required_by": required_by,
})
# Get dependencies
deps = db.query(ArtifactDependency).filter(
ArtifactDependency.artifact_id == artifact_id
).all()
# Resolve each dependency first (depth-first)
for dep in deps:
resolved_dep = _resolve_dependency_to_artifact(
db,
dep.dependency_project,
dep.dependency_package,
dep.version_constraint,
dep.tag_constraint,
)
if not resolved_dep:
constraint = dep.version_constraint or dep.tag_constraint
raise DependencyNotFoundError(
dep.dependency_project,
dep.dependency_package,
constraint,
)
dep_artifact_id, dep_version, dep_size = resolved_dep
_resolve_recursive(
dep_artifact_id,
dep.dependency_project,
dep.dependency_package,
dep_version,
dep_size,
pkg_key,
depth + 1,
)
visiting.remove(artifact_id)
visited.add(artifact_id)
# Add to resolution order (dependencies before dependents)
resolution_order.append(artifact_id)
# Store resolved artifact info
resolved_artifacts[artifact_id] = ResolvedArtifact(
artifact_id=artifact_id,
project=proj_name,
package=pkg_name,
version=version_or_tag,
size=size,
download_url=f"{base_url}/api/v1/project/{proj_name}/{pkg_name}/+/{version_or_tag}",
)
# Start resolution from root
_resolve_recursive(
root_artifact_id,
project_name,
package_name,
root_version,
root_size,
None,
)
# Build response in topological order
resolved_list = [resolved_artifacts[aid] for aid in resolution_order]
total_size = sum(r.size for r in resolved_list)
return DependencyResolutionResponse(
requested={
"project": project_name,
"package": package_name,
"ref": ref,
},
resolved=resolved_list,
total_size=total_size,
artifact_count=len(resolved_list),
)

View File

@@ -245,9 +245,10 @@ def extract_tarball_metadata(file: BinaryIO, filename: str) -> Dict[str, Any]:
break
# Try to split name and version
# Handle optional 'v' prefix on version (e.g., package-v1.0.0)
patterns = [
r"^(.+)-(\d+\.\d+(?:\.\d+)?(?:[-._]\w+)?)$", # name-version
r"^(.+)_(\d+\.\d+(?:\.\d+)?(?:[-._]\w+)?)$", # name_version
r"^(.+)-v?(\d+\.\d+(?:\.\d+)?(?:[-_]\w+)?)$", # name-version or name-vversion
r"^(.+)_v?(\d+\.\d+(?:\.\d+)?(?:[-_]\w+)?)$", # name_version or name_vversion
]
for pattern in patterns:

View File

@@ -72,6 +72,9 @@ class Package(Base):
consumers = relationship(
"Consumer", back_populates="package", cascade="all, delete-orphan"
)
versions = relationship(
"PackageVersion", back_populates="package", cascade="all, delete-orphan"
)
__table_args__ = (
Index("idx_packages_project_id", "project_id"),
@@ -113,6 +116,10 @@ class Artifact(Base):
tags = relationship("Tag", back_populates="artifact")
uploads = relationship("Upload", back_populates="artifact")
versions = relationship("PackageVersion", back_populates="artifact")
dependencies = relationship(
"ArtifactDependency", back_populates="artifact", cascade="all, delete-orphan"
)
@property
def sha256(self) -> str:
@@ -197,6 +204,38 @@ class TagHistory(Base):
)
class PackageVersion(Base):
"""Immutable version record for a package-artifact relationship.
Separates versions (immutable, set at upload) from tags (mutable labels).
Each artifact in a package can have at most one version.
"""
__tablename__ = "package_versions"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
package_id = Column(
UUID(as_uuid=True),
ForeignKey("packages.id", ondelete="CASCADE"),
nullable=False,
)
artifact_id = Column(String(64), ForeignKey("artifacts.id"), nullable=False)
version = Column(String(255), nullable=False)
version_source = Column(String(50)) # 'explicit', 'filename', 'metadata', 'migrated_from_tag'
created_at = Column(DateTime(timezone=True), default=datetime.utcnow)
created_by = Column(String(255), nullable=False)
package = relationship("Package", back_populates="versions")
artifact = relationship("Artifact", back_populates="versions")
__table_args__ = (
Index("idx_package_versions_package_id", "package_id"),
Index("idx_package_versions_artifact_id", "artifact_id"),
Index("idx_package_versions_package_version", "package_id", "version", unique=True),
Index("idx_package_versions_package_artifact", "package_id", "artifact_id", unique=True),
)
class Upload(Base):
__tablename__ = "uploads"
@@ -471,3 +510,54 @@ class PackageHistory(Base):
Index("idx_package_history_changed_at", "changed_at"),
Index("idx_package_history_package_changed_at", "package_id", "changed_at"),
)
class ArtifactDependency(Base):
"""Dependency declared by an artifact on another package.
Each artifact can declare dependencies on other packages, specifying either
an exact version or a tag. This enables recursive dependency resolution.
"""
__tablename__ = "artifact_dependencies"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
artifact_id = Column(
String(64),
ForeignKey("artifacts.id", ondelete="CASCADE"),
nullable=False,
)
dependency_project = Column(String(255), nullable=False)
dependency_package = Column(String(255), nullable=False)
version_constraint = Column(String(255), nullable=True)
tag_constraint = Column(String(255), nullable=True)
created_at = Column(DateTime(timezone=True), default=datetime.utcnow)
# Relationship to the artifact that declares this dependency
artifact = relationship("Artifact", back_populates="dependencies")
__table_args__ = (
# Exactly one of version_constraint or tag_constraint must be set
CheckConstraint(
"(version_constraint IS NOT NULL AND tag_constraint IS NULL) OR "
"(version_constraint IS NULL AND tag_constraint IS NOT NULL)",
name="check_constraint_type",
),
# Each artifact can only depend on a specific project/package once
Index(
"idx_artifact_dependencies_artifact_id",
"artifact_id",
),
Index(
"idx_artifact_dependencies_target",
"dependency_project",
"dependency_package",
),
Index(
"idx_artifact_dependencies_unique",
"artifact_id",
"dependency_project",
"dependency_package",
unique=True,
),
)

File diff suppressed because it is too large Load Diff

View File

@@ -173,6 +173,7 @@ class TagResponse(BaseModel):
artifact_id: str
created_at: datetime
created_by: str
version: Optional[str] = None # Version of the artifact this tag points to
class Config:
from_attributes = True
@@ -187,6 +188,7 @@ class TagDetailResponse(BaseModel):
artifact_id: str
created_at: datetime
created_by: str
version: Optional[str] = None # Version of the artifact this tag points to
# Artifact metadata
artifact_size: int
artifact_content_type: Optional[str]
@@ -383,6 +385,7 @@ class GlobalTagResponse(BaseModel):
package_name: str
artifact_size: Optional[int] = None
artifact_content_type: Optional[str] = None
version: Optional[str] = None # Version of the artifact this tag points to
class Config:
from_attributes = True
@@ -396,6 +399,8 @@ class UploadResponse(BaseModel):
project: str
package: str
tag: Optional[str]
version: Optional[str] = None # Version assigned to this artifact
version_source: Optional[str] = None # How version was determined: 'explicit', 'filename', 'metadata'
checksum_md5: Optional[str] = None
checksum_sha1: Optional[str] = None
s3_etag: Optional[str] = None
@@ -407,6 +412,9 @@ class UploadResponse(BaseModel):
content_type: Optional[str] = None
original_name: Optional[str] = None
created_at: Optional[datetime] = None
# Upload metrics (Issue #43)
duration_ms: Optional[int] = None # Upload duration in milliseconds
throughput_mbps: Optional[float] = None # Upload throughput in MB/s
# Resumable upload schemas
@@ -418,6 +426,7 @@ class ResumableUploadInitRequest(BaseModel):
content_type: Optional[str] = None
size: int
tag: Optional[str] = None
version: Optional[str] = None # Explicit version (auto-detected if not provided)
@field_validator("expected_hash")
@classmethod
@@ -472,6 +481,21 @@ class ResumableUploadStatusResponse(BaseModel):
total_uploaded_bytes: int
class UploadProgressResponse(BaseModel):
"""Progress information for an in-flight upload"""
upload_id: str
status: str # 'in_progress', 'completed', 'failed', 'not_found'
bytes_uploaded: int = 0
bytes_total: Optional[int] = None
percent_complete: Optional[float] = None
parts_uploaded: int = 0
parts_total: Optional[int] = None
started_at: Optional[datetime] = None
elapsed_seconds: Optional[float] = None
throughput_mbps: Optional[float] = None
# Consumer schemas
class ConsumerResponse(BaseModel):
id: UUID
@@ -484,6 +508,35 @@ class ConsumerResponse(BaseModel):
from_attributes = True
# Package version schemas
class PackageVersionResponse(BaseModel):
"""Immutable version record for an artifact in a package"""
id: UUID
package_id: UUID
artifact_id: str
version: str
version_source: Optional[str] = None # 'explicit', 'filename', 'metadata', 'migrated_from_tag'
created_at: datetime
created_by: str
# Enriched fields from joins
size: Optional[int] = None
content_type: Optional[str] = None
original_name: Optional[str] = None
tags: List[str] = [] # Tag names pointing to this artifact
class Config:
from_attributes = True
class PackageVersionDetailResponse(PackageVersionResponse):
"""Version with full artifact metadata"""
format_metadata: Optional[Dict[str, Any]] = None
checksum_md5: Optional[str] = None
checksum_sha1: Optional[str] = None
# Global search schemas
class SearchResultProject(BaseModel):
"""Project result for global search"""
@@ -863,3 +916,140 @@ class ProjectWithAccessResponse(ProjectResponse):
"""Project response with user's access level"""
user_access_level: Optional[str] = None
# Artifact Dependency schemas
class DependencyCreate(BaseModel):
"""Schema for creating a dependency"""
project: str
package: str
version: Optional[str] = None
tag: Optional[str] = None
@field_validator('version', 'tag')
@classmethod
def validate_constraint(cls, v, info):
return v
def model_post_init(self, __context):
"""Validate that exactly one of version or tag is set"""
if self.version is None and self.tag is None:
raise ValueError("Either 'version' or 'tag' must be specified")
if self.version is not None and self.tag is not None:
raise ValueError("Cannot specify both 'version' and 'tag'")
class DependencyResponse(BaseModel):
"""Schema for dependency response"""
id: UUID
artifact_id: str
project: str
package: str
version: Optional[str] = None
tag: Optional[str] = None
created_at: datetime
class Config:
from_attributes = True
@classmethod
def from_orm_model(cls, dep) -> "DependencyResponse":
"""Create from ORM model with field mapping"""
return cls(
id=dep.id,
artifact_id=dep.artifact_id,
project=dep.dependency_project,
package=dep.dependency_package,
version=dep.version_constraint,
tag=dep.tag_constraint,
created_at=dep.created_at,
)
class ArtifactDependenciesResponse(BaseModel):
"""Response containing all dependencies for an artifact"""
artifact_id: str
dependencies: List[DependencyResponse]
class DependentInfo(BaseModel):
"""Information about an artifact that depends on a package"""
artifact_id: str
project: str
package: str
version: Optional[str] = None
constraint_type: str # 'version' or 'tag'
constraint_value: str
class ReverseDependenciesResponse(BaseModel):
"""Response containing packages that depend on a given package"""
project: str
package: str
dependents: List[DependentInfo]
pagination: PaginationMeta
class EnsureFileDependency(BaseModel):
"""Dependency entry from orchard.ensure file"""
project: str
package: str
version: Optional[str] = None
tag: Optional[str] = None
@field_validator('version', 'tag')
@classmethod
def validate_constraint(cls, v, info):
return v
def model_post_init(self, __context):
"""Validate that exactly one of version or tag is set"""
if self.version is None and self.tag is None:
raise ValueError("Either 'version' or 'tag' must be specified")
if self.version is not None and self.tag is not None:
raise ValueError("Cannot specify both 'version' and 'tag'")
class EnsureFileContent(BaseModel):
"""Parsed content of orchard.ensure file"""
dependencies: List[EnsureFileDependency] = []
class ResolvedArtifact(BaseModel):
"""A resolved artifact in the dependency tree"""
artifact_id: str
project: str
package: str
version: Optional[str] = None
tag: Optional[str] = None
size: int
download_url: str
class DependencyResolutionResponse(BaseModel):
"""Response from dependency resolution endpoint"""
requested: Dict[str, str] # project, package, ref
resolved: List[ResolvedArtifact]
total_size: int
artifact_count: int
class DependencyConflict(BaseModel):
"""Details about a dependency conflict"""
project: str
package: str
requirements: List[Dict[str, Any]] # version/tag and required_by info
class DependencyConflictError(BaseModel):
"""Error response for dependency conflicts"""
error: str = "dependency_conflict"
message: str
conflicts: List[DependencyConflict]
class CircularDependencyError(BaseModel):
"""Error response for circular dependencies"""
error: str = "circular_dependency"
message: str
cycle: List[str] # List of "project/package" strings showing the cycle

View File

@@ -5,7 +5,7 @@ import hashlib
import logging
from sqlalchemy.orm import Session
from .models import Project, Package, Artifact, Tag, Upload
from .models import Project, Package, Artifact, Tag, Upload, PackageVersion, ArtifactDependency
from .storage import get_storage
logger = logging.getLogger(__name__)
@@ -74,7 +74,7 @@ TEST_PROJECTS = [
},
]
# Sample artifacts to create (content, tags)
# Sample artifacts to create (content, tags, version)
TEST_ARTIFACTS = [
{
"project": "frontend-libs",
@@ -83,6 +83,7 @@ TEST_ARTIFACTS = [
"filename": "ui-components-1.0.0.js",
"content_type": "application/javascript",
"tags": ["v1.0.0", "latest"],
"version": "1.0.0",
},
{
"project": "frontend-libs",
@@ -91,6 +92,7 @@ TEST_ARTIFACTS = [
"filename": "ui-components-1.1.0.js",
"content_type": "application/javascript",
"tags": ["v1.1.0"],
"version": "1.1.0",
},
{
"project": "frontend-libs",
@@ -99,6 +101,7 @@ TEST_ARTIFACTS = [
"filename": "tokens.json",
"content_type": "application/json",
"tags": ["v1.0.0", "latest"],
"version": "1.0.0",
},
{
"project": "backend-services",
@@ -107,6 +110,7 @@ TEST_ARTIFACTS = [
"filename": "utils-2.0.0.py",
"content_type": "text/x-python",
"tags": ["v2.0.0", "stable", "latest"],
"version": "2.0.0",
},
{
"project": "backend-services",
@@ -115,9 +119,21 @@ TEST_ARTIFACTS = [
"filename": "auth-lib-1.0.0.go",
"content_type": "text/x-go",
"tags": ["v1.0.0", "latest"],
"version": "1.0.0",
},
]
# Dependencies to create (source artifact -> dependency)
# Format: (source_project, source_package, source_version, dep_project, dep_package, version_constraint, tag_constraint)
TEST_DEPENDENCIES = [
# ui-components v1.1.0 depends on design-tokens v1.0.0
("frontend-libs", "ui-components", "1.1.0", "frontend-libs", "design-tokens", "1.0.0", None),
# auth-lib v1.0.0 depends on common-utils v2.0.0
("backend-services", "auth-lib", "1.0.0", "backend-services", "common-utils", "2.0.0", None),
# auth-lib v1.0.0 also depends on design-tokens (stable tag)
("backend-services", "auth-lib", "1.0.0", "frontend-libs", "design-tokens", None, "latest"),
]
def is_database_empty(db: Session) -> bool:
"""Check if the database has any projects."""
@@ -160,9 +176,10 @@ def seed_database(db: Session) -> None:
logger.info(f"Created {len(project_map)} projects and {len(package_map)} packages")
# Create artifacts and tags
# Create artifacts, tags, and versions
artifact_count = 0
tag_count = 0
version_count = 0
for artifact_data in TEST_ARTIFACTS:
project = project_map[artifact_data["project"]]
@@ -184,6 +201,11 @@ def seed_database(db: Session) -> None:
logger.warning(f"Failed to store artifact in S3: {e}")
continue
# Calculate ref_count: tags + version (if present)
ref_count = len(artifact_data["tags"])
if artifact_data.get("version"):
ref_count += 1
# Create artifact record
artifact = Artifact(
id=sha256_hash,
@@ -192,7 +214,7 @@ def seed_database(db: Session) -> None:
original_name=artifact_data["filename"],
created_by="seed-user",
s3_key=s3_key,
ref_count=len(artifact_data["tags"]),
ref_count=ref_count,
)
db.add(artifact)
@@ -206,6 +228,18 @@ def seed_database(db: Session) -> None:
db.add(upload)
artifact_count += 1
# Create version record if specified
if artifact_data.get("version"):
version = PackageVersion(
package_id=package.id,
artifact_id=sha256_hash,
version=artifact_data["version"],
version_source="explicit",
created_by="seed-user",
)
db.add(version)
version_count += 1
# Create tags
for tag_name in artifact_data["tags"]:
tag = Tag(
@@ -217,6 +251,40 @@ def seed_database(db: Session) -> None:
db.add(tag)
tag_count += 1
db.flush()
# Create dependencies
dependency_count = 0
for dep_data in TEST_DEPENDENCIES:
src_project, src_package, src_version, dep_project, dep_package, version_constraint, tag_constraint = dep_data
# Find the source artifact by looking up its version
src_pkg = package_map.get((src_project, src_package))
if not src_pkg:
logger.warning(f"Source package not found: {src_project}/{src_package}")
continue
# Find the artifact for this version
src_version_record = db.query(PackageVersion).filter(
PackageVersion.package_id == src_pkg.id,
PackageVersion.version == src_version,
).first()
if not src_version_record:
logger.warning(f"Source version not found: {src_project}/{src_package}@{src_version}")
continue
# Create the dependency
dependency = ArtifactDependency(
artifact_id=src_version_record.artifact_id,
dependency_project=dep_project,
dependency_package=dep_package,
version_constraint=version_constraint,
tag_constraint=tag_constraint,
)
db.add(dependency)
dependency_count += 1
db.commit()
logger.info(f"Created {artifact_count} artifacts and {tag_count} tags")
logger.info(f"Created {artifact_count} artifacts, {tag_count} tags, {version_count} versions, and {dependency_count} dependencies")
logger.info("Database seeding complete")

View File

@@ -242,15 +242,19 @@ class S3Storage:
},
)
self.client = boto3.client(
"s3",
endpoint_url=settings.s3_endpoint if settings.s3_endpoint else None,
region_name=settings.s3_region,
aws_access_key_id=settings.s3_access_key_id,
aws_secret_access_key=settings.s3_secret_access_key,
config=config,
verify=settings.s3_verify_ssl, # SSL/TLS verification
)
# Build client kwargs - only include credentials if explicitly provided
# This allows IRSA/IAM role credentials to be used when no explicit creds are set
client_kwargs = {
"endpoint_url": settings.s3_endpoint if settings.s3_endpoint else None,
"region_name": settings.s3_region,
"config": config,
"verify": settings.s3_verify_ssl,
}
if settings.s3_access_key_id and settings.s3_secret_access_key:
client_kwargs["aws_access_key_id"] = settings.s3_access_key_id
client_kwargs["aws_secret_access_key"] = settings.s3_secret_access_key
self.client = boto3.client("s3", **client_kwargs)
self.bucket = settings.s3_bucket
# Store active multipart uploads for resumable support
self._active_uploads: Dict[str, Dict[str, Any]] = {}
@@ -378,10 +382,16 @@ class S3Storage:
"""
# First pass: compute all hashes by streaming through file
try:
import time
sha256_hasher = hashlib.sha256()
md5_hasher = hashlib.md5()
sha1_hasher = hashlib.sha1()
size = 0
hash_start_time = time.time()
last_log_time = hash_start_time
log_interval_seconds = 5 # Log progress every 5 seconds
logger.info(f"Computing hashes for large file: expected_size={content_length}")
# Read file in chunks to compute hashes
while True:
@@ -393,6 +403,18 @@ class S3Storage:
sha1_hasher.update(chunk)
size += len(chunk)
# Log hash computation progress periodically
current_time = time.time()
if current_time - last_log_time >= log_interval_seconds:
elapsed = current_time - hash_start_time
percent = (size / content_length) * 100 if content_length > 0 else 0
throughput = (size / (1024 * 1024)) / elapsed if elapsed > 0 else 0
logger.info(
f"Hash computation progress: bytes={size}/{content_length} ({percent:.1f}%) "
f"throughput={throughput:.2f}MB/s"
)
last_log_time = current_time
# Enforce file size limit during streaming (protection against spoofing)
if size > settings.max_file_size:
raise FileSizeExceededError(
@@ -405,6 +427,14 @@ class S3Storage:
sha256_hash = sha256_hasher.hexdigest()
md5_hash = md5_hasher.hexdigest()
sha1_hash = sha1_hasher.hexdigest()
# Log hash computation completion
hash_elapsed = time.time() - hash_start_time
hash_throughput = (size / (1024 * 1024)) / hash_elapsed if hash_elapsed > 0 else 0
logger.info(
f"Hash computation completed: hash={sha256_hash[:16]}... "
f"size={size} duration={hash_elapsed:.2f}s throughput={hash_throughput:.2f}MB/s"
)
except (HashComputationError, FileSizeExceededError):
raise
except Exception as e:
@@ -458,8 +488,19 @@ class S3Storage:
upload_id = mpu["UploadId"]
try:
import time
parts = []
part_number = 1
bytes_uploaded = 0
upload_start_time = time.time()
last_log_time = upload_start_time
log_interval_seconds = 5 # Log progress every 5 seconds
total_parts = (content_length + MULTIPART_CHUNK_SIZE - 1) // MULTIPART_CHUNK_SIZE
logger.info(
f"Starting multipart upload: hash={sha256_hash[:16]}... "
f"size={content_length} parts={total_parts}"
)
while True:
chunk = file.read(MULTIPART_CHUNK_SIZE)
@@ -479,8 +520,32 @@ class S3Storage:
"ETag": response["ETag"],
}
)
bytes_uploaded += len(chunk)
# Log progress periodically
current_time = time.time()
if current_time - last_log_time >= log_interval_seconds:
elapsed = current_time - upload_start_time
percent = (bytes_uploaded / content_length) * 100
throughput = (bytes_uploaded / (1024 * 1024)) / elapsed if elapsed > 0 else 0
logger.info(
f"Upload progress: hash={sha256_hash[:16]}... "
f"part={part_number}/{total_parts} "
f"bytes={bytes_uploaded}/{content_length} ({percent:.1f}%) "
f"throughput={throughput:.2f}MB/s"
)
last_log_time = current_time
part_number += 1
# Log completion
total_elapsed = time.time() - upload_start_time
final_throughput = (content_length / (1024 * 1024)) / total_elapsed if total_elapsed > 0 else 0
logger.info(
f"Multipart upload completed: hash={sha256_hash[:16]}... "
f"size={content_length} duration={total_elapsed:.2f}s throughput={final_throughput:.2f}MB/s"
)
# Complete multipart upload
complete_response = self.client.complete_multipart_upload(
Bucket=self.bucket,
@@ -502,12 +567,28 @@ class S3Storage:
except Exception as e:
# Abort multipart upload on failure
logger.error(f"Multipart upload failed: {e}")
self.client.abort_multipart_upload(
Bucket=self.bucket,
Key=s3_key,
UploadId=upload_id,
error_str = str(e).lower()
is_client_disconnect = (
isinstance(e, (ConnectionResetError, BrokenPipeError)) or
"connection" in error_str or "broken pipe" in error_str or "reset" in error_str
)
if is_client_disconnect:
logger.warning(
f"Multipart upload aborted (client disconnect): hash={sha256_hash[:16]}... "
f"parts_uploaded={len(parts)} bytes_uploaded={bytes_uploaded}"
)
else:
logger.error(f"Multipart upload failed: hash={sha256_hash[:16]}... error={e}")
try:
self.client.abort_multipart_upload(
Bucket=self.bucket,
Key=s3_key,
UploadId=upload_id,
)
logger.info(f"Multipart upload aborted and cleaned up: upload_id={upload_id[:16]}...")
except Exception as abort_error:
logger.error(f"Failed to abort multipart upload: {abort_error}")
raise
def initiate_resumable_upload(self, expected_hash: str) -> Dict[str, Any]:
@@ -529,12 +610,17 @@ class S3Storage:
mpu = self.client.create_multipart_upload(Bucket=self.bucket, Key=s3_key)
upload_id = mpu["UploadId"]
import time
session = {
"upload_id": upload_id,
"s3_key": s3_key,
"already_exists": False,
"parts": [],
"expected_hash": expected_hash,
"started_at": time.time(),
"bytes_uploaded": 0,
"expected_size": None, # Set when init provides size
"status": "in_progress",
}
self._active_uploads[upload_id] = session
return session
@@ -561,10 +647,57 @@ class S3Storage:
part_info = {
"PartNumber": part_number,
"ETag": response["ETag"],
"size": len(data),
}
session["parts"].append(part_info)
session["bytes_uploaded"] = session.get("bytes_uploaded", 0) + len(data)
return part_info
def get_upload_progress(self, upload_id: str) -> Optional[Dict[str, Any]]:
"""
Get progress information for a resumable upload.
Returns None if upload not found.
"""
import time
session = self._active_uploads.get(upload_id)
if not session:
return None
bytes_uploaded = session.get("bytes_uploaded", 0)
expected_size = session.get("expected_size")
started_at = session.get("started_at")
progress = {
"upload_id": upload_id,
"status": session.get("status", "in_progress"),
"bytes_uploaded": bytes_uploaded,
"bytes_total": expected_size,
"parts_uploaded": len(session.get("parts", [])),
"parts_total": None,
"started_at": started_at,
"elapsed_seconds": None,
"percent_complete": None,
"throughput_mbps": None,
}
if expected_size and expected_size > 0:
progress["percent_complete"] = round((bytes_uploaded / expected_size) * 100, 2)
progress["parts_total"] = (expected_size + MULTIPART_CHUNK_SIZE - 1) // MULTIPART_CHUNK_SIZE
if started_at:
elapsed = time.time() - started_at
progress["elapsed_seconds"] = round(elapsed, 2)
if elapsed > 0 and bytes_uploaded > 0:
progress["throughput_mbps"] = round((bytes_uploaded / (1024 * 1024)) / elapsed, 2)
return progress
def set_upload_expected_size(self, upload_id: str, size: int):
"""Set the expected size for an upload (for progress tracking)."""
session = self._active_uploads.get(upload_id)
if session:
session["expected_size"] = size
def complete_resumable_upload(self, upload_id: str) -> Tuple[str, str]:
"""
Complete a resumable upload.
@@ -702,6 +835,36 @@ class S3Storage:
except ClientError:
return False
def delete_all(self) -> int:
"""
Delete all objects in the bucket.
Returns:
Number of objects deleted
"""
deleted_count = 0
try:
paginator = self.client.get_paginator("list_objects_v2")
for page in paginator.paginate(Bucket=self.bucket):
objects = page.get("Contents", [])
if not objects:
continue
# Delete objects in batches of 1000 (S3 limit)
delete_keys = [{"Key": obj["Key"]} for obj in objects]
if delete_keys:
self.client.delete_objects(
Bucket=self.bucket, Delete={"Objects": delete_keys}
)
deleted_count += len(delete_keys)
logger.info(f"Deleted {len(delete_keys)} objects from S3")
logger.info(f"Total objects deleted from S3: {deleted_count}")
return deleted_count
except ClientError as e:
logger.error(f"Failed to delete all S3 objects: {e}")
raise
def generate_presigned_url(
self,
s3_key: str,

View File

@@ -12,6 +12,8 @@ markers =
unit: Unit tests (no external dependencies)
integration: Integration tests (require database/storage)
slow: Slow tests (skip with -m "not slow")
large: Large file tests (100MB+, skip with -m "not large")
concurrent: Concurrent operation tests
# Coverage configuration
[coverage:run]

View File

@@ -9,6 +9,37 @@ This module provides:
import os
import pytest
# =============================================================================
# Pytest Markers
# =============================================================================
def pytest_configure(config):
"""Register custom pytest markers."""
config.addinivalue_line(
"markers",
"auth_intensive: marks tests that make many login requests (excluded from CI integration tests due to rate limiting)",
)
config.addinivalue_line(
"markers",
"integration: marks tests as integration tests",
)
config.addinivalue_line(
"markers",
"large: marks tests that handle large files (slow)",
)
config.addinivalue_line(
"markers",
"slow: marks tests as slow running",
)
config.addinivalue_line(
"markers",
"requires_direct_s3: marks tests that require direct S3/MinIO access (skipped in CI where S3 is not directly accessible)",
)
import io
from typing import Generator
from unittest.mock import MagicMock
@@ -32,6 +63,8 @@ from tests.factories import (
compute_md5,
compute_sha1,
upload_test_file,
generate_content,
generate_content_with_hash,
TEST_CONTENT_HELLO,
TEST_HASH_HELLO,
TEST_MD5_HELLO,
@@ -179,29 +212,64 @@ def test_app():
# =============================================================================
@pytest.fixture
@pytest.fixture(scope="session")
def integration_client():
"""
Create an authenticated test client for integration tests.
Uses the real database and MinIO from docker-compose.local.yml.
Authenticates as admin for write operations.
Uses the real database and MinIO from docker-compose.local.yml or deployed environment.
Authenticates as admin for write operations. Session-scoped to reuse login across tests.
Environment variables:
ORCHARD_TEST_URL: Base URL of the Orchard server (default: http://localhost:8080)
ORCHARD_TEST_USERNAME: Admin username for authentication (default: admin)
ORCHARD_TEST_PASSWORD: Admin password for authentication (default: changeme123)
"""
from httpx import Client
import httpx
# Connect to the running orchard-server container
# Connect to the running orchard-server container or deployed environment
base_url = os.environ.get("ORCHARD_TEST_URL", "http://localhost:8080")
username = os.environ.get("ORCHARD_TEST_USERNAME", "admin")
password = os.environ.get("ORCHARD_TEST_PASSWORD", "changeme123")
with Client(base_url=base_url, timeout=30.0) as client:
with httpx.Client(base_url=base_url, timeout=30.0) as client:
# Login as admin to enable write operations
login_response = client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
json={"username": username, "password": password},
)
# If login fails, tests will fail - that's expected if auth is broken
if login_response.status_code != 200:
# Try to continue without auth for backward compatibility
pass
pytest.fail(
f"Authentication failed against {base_url}: {login_response.status_code} - {login_response.text}. "
f"Set ORCHARD_TEST_USERNAME and ORCHARD_TEST_PASSWORD environment variables if using non-default credentials."
)
# Verify cookie was set
if not client.cookies:
pytest.fail(
f"Login succeeded but no session cookie was set. Response headers: {login_response.headers}"
)
yield client
@pytest.fixture
def auth_client():
"""
Create a function-scoped test client for authentication tests.
Unlike integration_client (session-scoped), this creates a fresh client
for each test. Use this for tests that manipulate authentication state
(login, logout, cookie clearing) to avoid polluting other tests.
Environment variables:
ORCHARD_TEST_URL: Base URL of the Orchard server (default: http://localhost:8080)
"""
import httpx
base_url = os.environ.get("ORCHARD_TEST_URL", "http://localhost:8080")
with httpx.Client(base_url=base_url, timeout=30.0) as client:
yield client
@@ -271,3 +339,41 @@ def test_content():
content = f"test-content-{uuid.uuid4().hex}".encode()
sha256 = compute_sha256(content)
return (content, sha256)
@pytest.fixture
def sized_content():
"""
Factory fixture for generating content of specific sizes.
Usage:
def test_example(sized_content):
content, hash = sized_content(1024) # 1KB
content, hash = sized_content(1024 * 1024) # 1MB
"""
def _generate(size: int, seed: int = None):
return generate_content_with_hash(size, seed)
return _generate
# =============================================================================
# Size Constants for Tests
# =============================================================================
# Common file sizes for boundary testing
SIZE_1B = 1
SIZE_1KB = 1024
SIZE_10KB = 10 * 1024
SIZE_100KB = 100 * 1024
SIZE_1MB = 1024 * 1024
SIZE_5MB = 5 * 1024 * 1024
SIZE_10MB = 10 * 1024 * 1024
SIZE_50MB = 50 * 1024 * 1024
SIZE_100MB = 100 * 1024 * 1024
SIZE_250MB = 250 * 1024 * 1024
SIZE_500MB = 500 * 1024 * 1024
SIZE_1GB = 1024 * 1024 * 1024
# Chunk size boundaries (based on typical S3 multipart chunk sizes)
CHUNK_SIZE = 64 * 1024 # 64KB typical chunk
MULTIPART_THRESHOLD = 100 * 1024 * 1024 # 100MB multipart threshold

View File

@@ -97,6 +97,7 @@ def upload_test_file(
content: bytes,
filename: str = "test.bin",
tag: Optional[str] = None,
version: Optional[str] = None,
) -> dict:
"""
Helper function to upload a test file via the API.
@@ -108,6 +109,7 @@ def upload_test_file(
content: File content as bytes
filename: Original filename
tag: Optional tag to assign
version: Optional version to assign
Returns:
The upload response as a dict
@@ -116,6 +118,8 @@ def upload_test_file(
data = {}
if tag:
data["tag"] = tag
if version:
data["version"] = version
response = client.post(
f"/api/v1/project/{project}/{package}/upload",
@@ -126,6 +130,41 @@ def upload_test_file(
return response.json()
def generate_content(size: int, seed: Optional[int] = None) -> bytes:
"""
Generate deterministic or random content of a specified size.
Args:
size: Size of content in bytes
seed: Optional seed for reproducible content (None for random)
Returns:
Bytes of the specified size
"""
if size == 0:
return b""
if seed is not None:
import random
rng = random.Random(seed)
return bytes(rng.randint(0, 255) for _ in range(size))
return os.urandom(size)
def generate_content_with_hash(size: int, seed: Optional[int] = None) -> tuple[bytes, str]:
"""
Generate content of specified size and compute its SHA256 hash.
Args:
size: Size of content in bytes
seed: Optional seed for reproducible content
Returns:
Tuple of (content_bytes, sha256_hash)
"""
content = generate_content(size, seed)
return content, compute_sha256(content)
# =============================================================================
# Project/Package Factories
# =============================================================================

View File

@@ -1,16 +1,25 @@
"""Integration tests for authentication API endpoints."""
"""Integration tests for authentication API endpoints.
Note: These tests are marked as auth_intensive because they make many login
requests. Dev/stage deployments have relaxed rate limits (1000/minute) to
allow these tests to run. Production uses strict rate limits (5/minute).
"""
import pytest
from uuid import uuid4
# Mark all tests in this module as auth_intensive (informational, not excluded from CI)
pytestmark = pytest.mark.auth_intensive
class TestAuthLogin:
"""Tests for login endpoint."""
@pytest.mark.integration
def test_login_success(self, integration_client):
def test_login_success(self, auth_client):
"""Test successful login with default admin credentials."""
response = integration_client.post(
response = auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
@@ -21,9 +30,9 @@ class TestAuthLogin:
assert "orchard_session" in response.cookies
@pytest.mark.integration
def test_login_invalid_password(self, integration_client):
def test_login_invalid_password(self, auth_client):
"""Test login with wrong password."""
response = integration_client.post(
response = auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "wrongpassword"},
)
@@ -31,9 +40,9 @@ class TestAuthLogin:
assert "Invalid username or password" in response.json()["detail"]
@pytest.mark.integration
def test_login_nonexistent_user(self, integration_client):
def test_login_nonexistent_user(self, auth_client):
"""Test login with non-existent user."""
response = integration_client.post(
response = auth_client.post(
"/api/v1/auth/login",
json={"username": "nonexistent", "password": "password"},
)
@@ -44,24 +53,24 @@ class TestAuthLogout:
"""Tests for logout endpoint."""
@pytest.mark.integration
def test_logout_success(self, integration_client):
def test_logout_success(self, auth_client):
"""Test successful logout."""
# First login
login_response = integration_client.post(
login_response = auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
assert login_response.status_code == 200
# Then logout
logout_response = integration_client.post("/api/v1/auth/logout")
logout_response = auth_client.post("/api/v1/auth/logout")
assert logout_response.status_code == 200
assert "Logged out successfully" in logout_response.json()["message"]
@pytest.mark.integration
def test_logout_without_session(self, integration_client):
def test_logout_without_session(self, auth_client):
"""Test logout without being logged in."""
response = integration_client.post("/api/v1/auth/logout")
response = auth_client.post("/api/v1/auth/logout")
# Should succeed even without session
assert response.status_code == 200
@@ -70,15 +79,15 @@ class TestAuthMe:
"""Tests for get current user endpoint."""
@pytest.mark.integration
def test_get_me_authenticated(self, integration_client):
def test_get_me_authenticated(self, auth_client):
"""Test getting current user when authenticated."""
# Login first
integration_client.post(
auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
response = integration_client.get("/api/v1/auth/me")
response = auth_client.get("/api/v1/auth/me")
assert response.status_code == 200
data = response.json()
assert data["username"] == "admin"
@@ -87,67 +96,88 @@ class TestAuthMe:
assert "created_at" in data
@pytest.mark.integration
def test_get_me_unauthenticated(self, integration_client):
def test_get_me_unauthenticated(self, auth_client):
"""Test getting current user without authentication."""
# Clear any existing cookies
integration_client.cookies.clear()
auth_client.cookies.clear()
response = integration_client.get("/api/v1/auth/me")
response = auth_client.get("/api/v1/auth/me")
assert response.status_code == 401
assert "Not authenticated" in response.json()["detail"]
class TestAuthChangePassword:
"""Tests for change password endpoint."""
"""Tests for change password endpoint.
Note: These tests use dedicated test users instead of admin to avoid
invalidating the integration_client session (which uses admin).
"""
@pytest.mark.integration
def test_change_password_success(self, integration_client):
def test_change_password_success(self, auth_client):
"""Test successful password change."""
# Login first
integration_client.post(
# Login as admin to create a test user
auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
test_username = f"pwchange_{uuid4().hex[:8]}"
auth_client.post(
"/api/v1/admin/users",
json={"username": test_username, "password": "oldpassword123"},
)
# Login as test user
auth_client.cookies.clear()
auth_client.post(
"/api/v1/auth/login",
json={"username": test_username, "password": "oldpassword123"},
)
# Change password
response = integration_client.post(
response = auth_client.post(
"/api/v1/auth/change-password",
json={"current_password": "changeme123", "new_password": "newpassword123"},
json={"current_password": "oldpassword123", "new_password": "newpassword123"},
)
assert response.status_code == 200
# Verify old password no longer works
integration_client.cookies.clear()
response = integration_client.post(
auth_client.cookies.clear()
response = auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
json={"username": test_username, "password": "oldpassword123"},
)
assert response.status_code == 401
# Verify new password works
response = integration_client.post(
response = auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "newpassword123"},
json={"username": test_username, "password": "newpassword123"},
)
assert response.status_code == 200
# Reset password back to original for other tests
reset_response = integration_client.post(
"/api/v1/auth/change-password",
json={"current_password": "newpassword123", "new_password": "changeme123"},
)
assert reset_response.status_code == 200, "Failed to reset admin password back to default"
@pytest.mark.integration
def test_change_password_wrong_current(self, integration_client):
def test_change_password_wrong_current(self, auth_client):
"""Test password change with wrong current password."""
# Login first
integration_client.post(
# Login as admin to create a test user
auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
test_username = f"pwwrong_{uuid4().hex[:8]}"
auth_client.post(
"/api/v1/admin/users",
json={"username": test_username, "password": "password123"},
)
response = integration_client.post(
# Login as test user
auth_client.cookies.clear()
auth_client.post(
"/api/v1/auth/login",
json={"username": test_username, "password": "password123"},
)
response = auth_client.post(
"/api/v1/auth/change-password",
json={"current_password": "wrongpassword", "new_password": "newpassword"},
)
@@ -159,16 +189,16 @@ class TestAPIKeys:
"""Tests for API key management endpoints."""
@pytest.mark.integration
def test_create_and_list_api_key(self, integration_client):
def test_create_and_list_api_key(self, auth_client):
"""Test creating and listing API keys."""
# Login first
integration_client.post(
auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
# Create API key
create_response = integration_client.post(
create_response = auth_client.post(
"/api/v1/auth/keys",
json={"name": "test-key", "description": "Test API key"},
)
@@ -182,23 +212,23 @@ class TestAPIKeys:
api_key = data["key"]
# List API keys
list_response = integration_client.get("/api/v1/auth/keys")
list_response = auth_client.get("/api/v1/auth/keys")
assert list_response.status_code == 200
keys = list_response.json()
assert any(k["id"] == key_id for k in keys)
# Clean up - delete the key
integration_client.delete(f"/api/v1/auth/keys/{key_id}")
auth_client.delete(f"/api/v1/auth/keys/{key_id}")
@pytest.mark.integration
def test_use_api_key_for_auth(self, integration_client):
def test_use_api_key_for_auth(self, auth_client):
"""Test using API key for authentication."""
# Login and create API key
integration_client.post(
auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
create_response = integration_client.post(
create_response = auth_client.post(
"/api/v1/auth/keys",
json={"name": "auth-test-key"},
)
@@ -206,8 +236,8 @@ class TestAPIKeys:
key_id = create_response.json()["id"]
# Clear cookies and use API key
integration_client.cookies.clear()
response = integration_client.get(
auth_client.cookies.clear()
response = auth_client.get(
"/api/v1/auth/me",
headers={"Authorization": f"Bearer {api_key}"},
)
@@ -215,21 +245,21 @@ class TestAPIKeys:
assert response.json()["username"] == "admin"
# Clean up
integration_client.post(
auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
integration_client.delete(f"/api/v1/auth/keys/{key_id}")
auth_client.delete(f"/api/v1/auth/keys/{key_id}")
@pytest.mark.integration
def test_delete_api_key(self, integration_client):
def test_delete_api_key(self, auth_client):
"""Test revoking an API key."""
# Login and create API key
integration_client.post(
auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
create_response = integration_client.post(
create_response = auth_client.post(
"/api/v1/auth/keys",
json={"name": "delete-test-key"},
)
@@ -237,12 +267,12 @@ class TestAPIKeys:
api_key = create_response.json()["key"]
# Delete the key
delete_response = integration_client.delete(f"/api/v1/auth/keys/{key_id}")
delete_response = auth_client.delete(f"/api/v1/auth/keys/{key_id}")
assert delete_response.status_code == 200
# Verify key no longer works
integration_client.cookies.clear()
response = integration_client.get(
auth_client.cookies.clear()
response = auth_client.get(
"/api/v1/auth/me",
headers={"Authorization": f"Bearer {api_key}"},
)
@@ -253,32 +283,32 @@ class TestAdminUserManagement:
"""Tests for admin user management endpoints."""
@pytest.mark.integration
def test_list_users(self, integration_client):
def test_list_users(self, auth_client):
"""Test listing users as admin."""
# Login as admin
integration_client.post(
auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
response = integration_client.get("/api/v1/admin/users")
response = auth_client.get("/api/v1/admin/users")
assert response.status_code == 200
users = response.json()
assert len(users) >= 1
assert any(u["username"] == "admin" for u in users)
@pytest.mark.integration
def test_create_user(self, integration_client):
def test_create_user(self, auth_client):
"""Test creating a new user as admin."""
# Login as admin
integration_client.post(
auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
# Create new user
test_username = f"testuser_{uuid4().hex[:8]}"
response = integration_client.post(
response = auth_client.post(
"/api/v1/admin/users",
json={
"username": test_username,
@@ -293,31 +323,31 @@ class TestAdminUserManagement:
assert data["is_admin"] is False
# Verify new user can login
integration_client.cookies.clear()
login_response = integration_client.post(
auth_client.cookies.clear()
login_response = auth_client.post(
"/api/v1/auth/login",
json={"username": test_username, "password": "testpassword"},
)
assert login_response.status_code == 200
@pytest.mark.integration
def test_update_user(self, integration_client):
def test_update_user(self, auth_client):
"""Test updating a user as admin."""
# Login as admin
integration_client.post(
auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
# Create a test user
test_username = f"updateuser_{uuid4().hex[:8]}"
integration_client.post(
auth_client.post(
"/api/v1/admin/users",
json={"username": test_username, "password": "password"},
)
# Update the user
response = integration_client.put(
response = auth_client.put(
f"/api/v1/admin/users/{test_username}",
json={"email": "updated@example.com", "is_admin": True},
)
@@ -327,59 +357,59 @@ class TestAdminUserManagement:
assert data["is_admin"] is True
@pytest.mark.integration
def test_reset_user_password(self, integration_client):
def test_reset_user_password(self, auth_client):
"""Test resetting a user's password as admin."""
# Login as admin
integration_client.post(
auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
# Create a test user
test_username = f"resetuser_{uuid4().hex[:8]}"
integration_client.post(
auth_client.post(
"/api/v1/admin/users",
json={"username": test_username, "password": "oldpassword"},
)
# Reset password
response = integration_client.post(
response = auth_client.post(
f"/api/v1/admin/users/{test_username}/reset-password",
json={"new_password": "newpassword"},
)
assert response.status_code == 200
# Verify new password works
integration_client.cookies.clear()
login_response = integration_client.post(
auth_client.cookies.clear()
login_response = auth_client.post(
"/api/v1/auth/login",
json={"username": test_username, "password": "newpassword"},
)
assert login_response.status_code == 200
@pytest.mark.integration
def test_non_admin_cannot_access_admin_endpoints(self, integration_client):
def test_non_admin_cannot_access_admin_endpoints(self, auth_client):
"""Test that non-admin users cannot access admin endpoints."""
# Login as admin and create non-admin user
integration_client.post(
auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
test_username = f"nonadmin_{uuid4().hex[:8]}"
integration_client.post(
auth_client.post(
"/api/v1/admin/users",
json={"username": test_username, "password": "password", "is_admin": False},
)
# Login as non-admin
integration_client.cookies.clear()
integration_client.post(
auth_client.cookies.clear()
auth_client.post(
"/api/v1/auth/login",
json={"username": test_username, "password": "password"},
)
# Try to access admin endpoints
response = integration_client.get("/api/v1/admin/users")
response = auth_client.get("/api/v1/admin/users")
assert response.status_code == 403
assert "Admin privileges required" in response.json()["detail"]
@@ -388,28 +418,28 @@ class TestSecurityEdgeCases:
"""Tests for security edge cases and validation."""
@pytest.mark.integration
def test_login_inactive_user(self, integration_client):
def test_login_inactive_user(self, auth_client):
"""Test that inactive users cannot login."""
# Login as admin and create a user
integration_client.post(
auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
test_username = f"inactive_{uuid4().hex[:8]}"
integration_client.post(
auth_client.post(
"/api/v1/admin/users",
json={"username": test_username, "password": "password123"},
)
# Deactivate the user
integration_client.put(
auth_client.put(
f"/api/v1/admin/users/{test_username}",
json={"is_active": False},
)
# Try to login as inactive user
integration_client.cookies.clear()
response = integration_client.post(
auth_client.cookies.clear()
response = auth_client.post(
"/api/v1/auth/login",
json={"username": test_username, "password": "password123"},
)
@@ -417,14 +447,14 @@ class TestSecurityEdgeCases:
assert "Invalid username or password" in response.json()["detail"]
@pytest.mark.integration
def test_password_too_short_on_create(self, integration_client):
def test_password_too_short_on_create(self, auth_client):
"""Test that short passwords are rejected when creating users."""
integration_client.post(
auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
response = integration_client.post(
response = auth_client.post(
"/api/v1/admin/users",
json={"username": f"shortpw_{uuid4().hex[:8]}", "password": "short"},
)
@@ -432,36 +462,49 @@ class TestSecurityEdgeCases:
assert "at least 8 characters" in response.json()["detail"]
@pytest.mark.integration
def test_password_too_short_on_change(self, integration_client):
def test_password_too_short_on_change(self, auth_client):
"""Test that short passwords are rejected when changing password."""
integration_client.post(
# Create test user
auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
test_username = f"shortchange_{uuid4().hex[:8]}"
auth_client.post(
"/api/v1/admin/users",
json={"username": test_username, "password": "password123"},
)
response = integration_client.post(
# Login as test user
auth_client.cookies.clear()
auth_client.post(
"/api/v1/auth/login",
json={"username": test_username, "password": "password123"},
)
response = auth_client.post(
"/api/v1/auth/change-password",
json={"current_password": "changeme123", "new_password": "short"},
json={"current_password": "password123", "new_password": "short"},
)
assert response.status_code == 400
assert "at least 8 characters" in response.json()["detail"]
@pytest.mark.integration
def test_password_too_short_on_reset(self, integration_client):
def test_password_too_short_on_reset(self, auth_client):
"""Test that short passwords are rejected when resetting password."""
integration_client.post(
auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
# Create a test user first
test_username = f"resetshort_{uuid4().hex[:8]}"
integration_client.post(
auth_client.post(
"/api/v1/admin/users",
json={"username": test_username, "password": "password123"},
)
response = integration_client.post(
response = auth_client.post(
f"/api/v1/admin/users/{test_username}/reset-password",
json={"new_password": "short"},
)
@@ -469,23 +512,23 @@ class TestSecurityEdgeCases:
assert "at least 8 characters" in response.json()["detail"]
@pytest.mark.integration
def test_duplicate_username_rejected(self, integration_client):
def test_duplicate_username_rejected(self, auth_client):
"""Test that duplicate usernames are rejected."""
integration_client.post(
auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
test_username = f"duplicate_{uuid4().hex[:8]}"
# Create user first time
response1 = integration_client.post(
response1 = auth_client.post(
"/api/v1/admin/users",
json={"username": test_username, "password": "password123"},
)
assert response1.status_code == 200
# Try to create same username again
response2 = integration_client.post(
response2 = auth_client.post(
"/api/v1/admin/users",
json={"username": test_username, "password": "password456"},
)
@@ -493,14 +536,14 @@ class TestSecurityEdgeCases:
assert "already exists" in response2.json()["detail"]
@pytest.mark.integration
def test_cannot_delete_other_users_api_key(self, integration_client):
def test_cannot_delete_other_users_api_key(self, auth_client):
"""Test that users cannot delete API keys owned by other users."""
# Login as admin and create an API key
integration_client.post(
auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
create_response = integration_client.post(
create_response = auth_client.post(
"/api/v1/auth/keys",
json={"name": "admin-key"},
)
@@ -508,253 +551,65 @@ class TestSecurityEdgeCases:
# Create a non-admin user
test_username = f"nonadmin_{uuid4().hex[:8]}"
integration_client.post(
auth_client.post(
"/api/v1/admin/users",
json={"username": test_username, "password": "password123"},
)
# Login as non-admin
integration_client.cookies.clear()
integration_client.post(
auth_client.cookies.clear()
auth_client.post(
"/api/v1/auth/login",
json={"username": test_username, "password": "password123"},
)
# Try to delete admin's API key
response = integration_client.delete(f"/api/v1/auth/keys/{admin_key_id}")
response = auth_client.delete(f"/api/v1/auth/keys/{admin_key_id}")
assert response.status_code == 403
assert "Cannot delete another user's API key" in response.json()["detail"]
# Cleanup: login as admin and delete the key
integration_client.cookies.clear()
integration_client.post(
auth_client.cookies.clear()
auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
integration_client.delete(f"/api/v1/auth/keys/{admin_key_id}")
auth_client.delete(f"/api/v1/auth/keys/{admin_key_id}")
@pytest.mark.integration
def test_sessions_invalidated_on_password_change(self, integration_client):
def test_sessions_invalidated_on_password_change(self, auth_client):
"""Test that all sessions are invalidated when password is changed."""
# Create a test user
integration_client.post(
auth_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
test_username = f"sessiontest_{uuid4().hex[:8]}"
integration_client.post(
auth_client.post(
"/api/v1/admin/users",
json={"username": test_username, "password": "password123"},
)
# Login as test user
integration_client.cookies.clear()
login_response = integration_client.post(
auth_client.cookies.clear()
login_response = auth_client.post(
"/api/v1/auth/login",
json={"username": test_username, "password": "password123"},
)
assert login_response.status_code == 200
# Verify session works
me_response = integration_client.get("/api/v1/auth/me")
me_response = auth_client.get("/api/v1/auth/me")
assert me_response.status_code == 200
# Change password
integration_client.post(
auth_client.post(
"/api/v1/auth/change-password",
json={"current_password": "password123", "new_password": "newpassword123"},
)
# Old session should be invalidated - try to access /me
# (note: the change-password call itself may have cleared the session cookie)
me_response2 = integration_client.get("/api/v1/auth/me")
# This should fail because all sessions were invalidated
assert me_response2.status_code == 401
class TestSecurityEdgeCases:
"""Tests for security edge cases and validation."""
@pytest.mark.integration
def test_login_inactive_user(self, integration_client):
"""Test that inactive users cannot login."""
# Login as admin and create a user
integration_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
test_username = f"inactive_{uuid4().hex[:8]}"
integration_client.post(
"/api/v1/admin/users",
json={"username": test_username, "password": "password123"},
)
# Deactivate the user
integration_client.put(
f"/api/v1/admin/users/{test_username}",
json={"is_active": False},
)
# Try to login as inactive user
integration_client.cookies.clear()
response = integration_client.post(
"/api/v1/auth/login",
json={"username": test_username, "password": "password123"},
)
assert response.status_code == 401
assert "Invalid username or password" in response.json()["detail"]
@pytest.mark.integration
def test_password_too_short_on_create(self, integration_client):
"""Test that short passwords are rejected when creating users."""
integration_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
response = integration_client.post(
"/api/v1/admin/users",
json={"username": f"shortpw_{uuid4().hex[:8]}", "password": "short"},
)
assert response.status_code == 400
assert "at least 8 characters" in response.json()["detail"]
@pytest.mark.integration
def test_password_too_short_on_change(self, integration_client):
"""Test that short passwords are rejected when changing password."""
integration_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
response = integration_client.post(
"/api/v1/auth/change-password",
json={"current_password": "changeme123", "new_password": "short"},
)
assert response.status_code == 400
assert "at least 8 characters" in response.json()["detail"]
@pytest.mark.integration
def test_password_too_short_on_reset(self, integration_client):
"""Test that short passwords are rejected when resetting password."""
integration_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
# Create a test user first
test_username = f"resetshort_{uuid4().hex[:8]}"
integration_client.post(
"/api/v1/admin/users",
json={"username": test_username, "password": "password123"},
)
response = integration_client.post(
f"/api/v1/admin/users/{test_username}/reset-password",
json={"new_password": "short"},
)
assert response.status_code == 400
assert "at least 8 characters" in response.json()["detail"]
@pytest.mark.integration
def test_duplicate_username_rejected(self, integration_client):
"""Test that duplicate usernames are rejected."""
integration_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
test_username = f"duplicate_{uuid4().hex[:8]}"
# Create user first time
response1 = integration_client.post(
"/api/v1/admin/users",
json={"username": test_username, "password": "password123"},
)
assert response1.status_code == 200
# Try to create same username again
response2 = integration_client.post(
"/api/v1/admin/users",
json={"username": test_username, "password": "password456"},
)
assert response2.status_code == 409
assert "already exists" in response2.json()["detail"]
@pytest.mark.integration
def test_cannot_delete_other_users_api_key(self, integration_client):
"""Test that users cannot delete API keys owned by other users."""
# Login as admin and create an API key
integration_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
create_response = integration_client.post(
"/api/v1/auth/keys",
json={"name": "admin-key"},
)
admin_key_id = create_response.json()["id"]
# Create a non-admin user
test_username = f"nonadmin_{uuid4().hex[:8]}"
integration_client.post(
"/api/v1/admin/users",
json={"username": test_username, "password": "password123"},
)
# Login as non-admin
integration_client.cookies.clear()
integration_client.post(
"/api/v1/auth/login",
json={"username": test_username, "password": "password123"},
)
# Try to delete admin's API key
response = integration_client.delete(f"/api/v1/auth/keys/{admin_key_id}")
assert response.status_code == 403
assert "Cannot delete another user's API key" in response.json()["detail"]
# Cleanup: login as admin and delete the key
integration_client.cookies.clear()
integration_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
integration_client.delete(f"/api/v1/auth/keys/{admin_key_id}")
@pytest.mark.integration
def test_sessions_invalidated_on_password_change(self, integration_client):
"""Test that all sessions are invalidated when password is changed."""
# Create a test user
integration_client.post(
"/api/v1/auth/login",
json={"username": "admin", "password": "changeme123"},
)
test_username = f"sessiontest_{uuid4().hex[:8]}"
integration_client.post(
"/api/v1/admin/users",
json={"username": test_username, "password": "password123"},
)
# Login as test user
integration_client.cookies.clear()
login_response = integration_client.post(
"/api/v1/auth/login",
json={"username": test_username, "password": "password123"},
)
assert login_response.status_code == 200
# Verify session works
me_response = integration_client.get("/api/v1/auth/me")
assert me_response.status_code == 200
# Change password
integration_client.post(
"/api/v1/auth/change-password",
json={"current_password": "password123", "new_password": "newpassword123"},
)
# Old session should be invalidated - try to access /me
# (note: the change-password call itself may have cleared the session cookie)
me_response2 = integration_client.get("/api/v1/auth/me")
me_response2 = auth_client.get("/api/v1/auth/me")
# This should fail because all sessions were invalidated
assert me_response2.status_code == 401

View File

@@ -0,0 +1,737 @@
"""
Integration tests for concurrent upload and download operations.
Tests cover:
- Concurrent uploads of different files
- Concurrent uploads of same file (deduplication race)
- Concurrent downloads of same artifact
- Concurrent downloads of different artifacts
- Mixed concurrent uploads and downloads
- Data corruption prevention under concurrency
"""
import pytest
import io
import os
from concurrent.futures import ThreadPoolExecutor, as_completed
from tests.factories import (
compute_sha256,
upload_test_file,
generate_content_with_hash,
)
def get_api_key(integration_client):
"""Create an API key for concurrent test workers."""
import uuid
response = integration_client.post(
"/api/v1/auth/keys",
json={"name": f"concurrent-test-{uuid.uuid4().hex[:8]}"},
)
if response.status_code == 200:
return response.json()["key"]
return None
class TestConcurrentUploads:
"""Tests for concurrent upload operations."""
@pytest.mark.integration
@pytest.mark.concurrent
def test_2_concurrent_uploads_different_files(self, integration_client, test_package):
"""Test 2 concurrent uploads of different files."""
project, package = test_package
api_key = get_api_key(integration_client)
assert api_key, "Failed to create API key"
files_data = [
generate_content_with_hash(1024, seed=i) for i in range(2)
]
results = []
errors = []
def upload_worker(idx, content, expected_hash):
try:
from httpx import Client
base_url = os.environ.get("ORCHARD_TEST_URL", "http://localhost:8080")
with Client(base_url=base_url, timeout=60.0) as client:
files = {
"file": (f"file-{idx}.bin", io.BytesIO(content), "application/octet-stream")
}
response = client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
data={"tag": f"concurrent-{idx}"},
headers={"Authorization": f"Bearer {api_key}"},
)
if response.status_code == 200:
result = response.json()
results.append((idx, result, expected_hash))
else:
errors.append(f"Worker {idx}: Status {response.status_code}: {response.text}")
except Exception as e:
errors.append(f"Worker {idx}: {str(e)}")
with ThreadPoolExecutor(max_workers=2) as executor:
futures = [
executor.submit(upload_worker, i, content, hash)
for i, (content, hash) in enumerate(files_data)
]
for future in as_completed(futures):
pass
assert len(errors) == 0, f"Errors: {errors}"
assert len(results) == 2
# Verify each upload returned correct artifact_id
for idx, result, expected_hash in results:
assert result["artifact_id"] == expected_hash
@pytest.mark.integration
@pytest.mark.concurrent
def test_5_concurrent_uploads_different_files(self, integration_client, test_package):
"""Test 5 concurrent uploads of different files."""
project, package = test_package
api_key = get_api_key(integration_client)
assert api_key, "Failed to create API key"
num_files = 5
files_data = [
generate_content_with_hash(2048, seed=100 + i) for i in range(num_files)
]
results = []
errors = []
def upload_worker(idx, content, expected_hash):
try:
from httpx import Client
base_url = os.environ.get("ORCHARD_TEST_URL", "http://localhost:8080")
with Client(base_url=base_url, timeout=60.0) as client:
files = {
"file": (f"file-{idx}.bin", io.BytesIO(content), "application/octet-stream")
}
response = client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
data={"tag": f"concurrent5-{idx}"},
headers={"Authorization": f"Bearer {api_key}"},
)
if response.status_code == 200:
result = response.json()
results.append((idx, result, expected_hash))
else:
errors.append(f"Worker {idx}: Status {response.status_code}")
except Exception as e:
errors.append(f"Worker {idx}: {str(e)}")
with ThreadPoolExecutor(max_workers=num_files) as executor:
futures = [
executor.submit(upload_worker, i, content, hash)
for i, (content, hash) in enumerate(files_data)
]
for future in as_completed(futures):
pass
assert len(errors) == 0, f"Errors: {errors}"
assert len(results) == num_files
# Verify all uploads have unique artifact_ids
artifact_ids = set(r[1]["artifact_id"] for r in results)
assert len(artifact_ids) == num_files
@pytest.mark.integration
@pytest.mark.concurrent
def test_10_concurrent_uploads_different_files(self, integration_client, test_package):
"""Test 10 concurrent uploads of different files."""
project, package = test_package
api_key = get_api_key(integration_client)
assert api_key, "Failed to create API key"
num_files = 10
files_data = [
generate_content_with_hash(1024, seed=200 + i) for i in range(num_files)
]
results = []
errors = []
def upload_worker(idx, content, expected_hash):
try:
from httpx import Client
base_url = os.environ.get("ORCHARD_TEST_URL", "http://localhost:8080")
with Client(base_url=base_url, timeout=60.0) as client:
files = {
"file": (f"file-{idx}.bin", io.BytesIO(content), "application/octet-stream")
}
response = client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
data={"tag": f"concurrent10-{idx}"},
headers={"Authorization": f"Bearer {api_key}"},
)
if response.status_code == 200:
result = response.json()
results.append((idx, result, expected_hash))
else:
errors.append(f"Worker {idx}: Status {response.status_code}")
except Exception as e:
errors.append(f"Worker {idx}: {str(e)}")
with ThreadPoolExecutor(max_workers=num_files) as executor:
futures = [
executor.submit(upload_worker, i, content, hash)
for i, (content, hash) in enumerate(files_data)
]
for future in as_completed(futures):
pass
assert len(errors) == 0, f"Errors: {errors}"
assert len(results) == num_files
@pytest.mark.integration
@pytest.mark.concurrent
def test_concurrent_uploads_same_file_deduplication(self, integration_client, test_package):
"""Test concurrent uploads of same file handle deduplication correctly."""
project, package = test_package
api_key = get_api_key(integration_client)
assert api_key, "Failed to create API key"
content, expected_hash = generate_content_with_hash(4096, seed=999)
num_concurrent = 5
results = []
errors = []
def upload_worker(idx):
try:
from httpx import Client
base_url = os.environ.get("ORCHARD_TEST_URL", "http://localhost:8080")
with Client(base_url=base_url, timeout=60.0) as client:
files = {
"file": (f"same-{idx}.bin", io.BytesIO(content), "application/octet-stream")
}
response = client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
data={"tag": f"dedup-{idx}"},
headers={"Authorization": f"Bearer {api_key}"},
)
if response.status_code == 200:
results.append(response.json())
else:
errors.append(f"Worker {idx}: Status {response.status_code}")
except Exception as e:
errors.append(f"Worker {idx}: {str(e)}")
with ThreadPoolExecutor(max_workers=num_concurrent) as executor:
futures = [executor.submit(upload_worker, i) for i in range(num_concurrent)]
for future in as_completed(futures):
pass
assert len(errors) == 0, f"Errors: {errors}"
assert len(results) == num_concurrent
# All should have same artifact_id
artifact_ids = set(r["artifact_id"] for r in results)
assert len(artifact_ids) == 1
assert expected_hash in artifact_ids
# Verify final ref_count equals number of uploads
response = integration_client.get(f"/api/v1/artifact/{expected_hash}")
assert response.status_code == 200
assert response.json()["ref_count"] == num_concurrent
@pytest.mark.integration
@pytest.mark.concurrent
def test_concurrent_uploads_to_different_packages(self, integration_client, test_project, unique_test_id):
"""Test concurrent uploads to different packages."""
project = test_project
api_key = get_api_key(integration_client)
assert api_key, "Failed to create API key"
num_packages = 3
package_names = []
# Create multiple packages
for i in range(num_packages):
pkg_name = f"pkg-{unique_test_id}-{i}"
response = integration_client.post(
f"/api/v1/project/{project}/packages",
json={"name": pkg_name, "description": f"Package {i}"},
)
assert response.status_code == 200
package_names.append(pkg_name)
files_data = [
generate_content_with_hash(1024, seed=300 + i) for i in range(num_packages)
]
results = []
errors = []
def upload_worker(idx, package, content, expected_hash):
try:
from httpx import Client
base_url = os.environ.get("ORCHARD_TEST_URL", "http://localhost:8080")
with Client(base_url=base_url, timeout=60.0) as client:
files = {
"file": (f"file-{idx}.bin", io.BytesIO(content), "application/octet-stream")
}
response = client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
data={"tag": "latest"},
headers={"Authorization": f"Bearer {api_key}"},
)
if response.status_code == 200:
result = response.json()
results.append((package, result, expected_hash))
else:
errors.append(f"Worker {idx}: Status {response.status_code}")
except Exception as e:
errors.append(f"Worker {idx}: {str(e)}")
with ThreadPoolExecutor(max_workers=num_packages) as executor:
futures = [
executor.submit(upload_worker, i, package_names[i], content, hash)
for i, (content, hash) in enumerate(files_data)
]
for future in as_completed(futures):
pass
assert len(errors) == 0, f"Errors: {errors}"
assert len(results) == num_packages
class TestConcurrentDownloads:
"""Tests for concurrent download operations."""
@pytest.mark.integration
@pytest.mark.concurrent
def test_2_concurrent_downloads_same_artifact(self, integration_client, test_package):
"""Test 2 concurrent downloads of same artifact."""
project, package = test_package
content, expected_hash = generate_content_with_hash(2048, seed=400)
# Upload first
upload_test_file(integration_client, project, package, content, tag="download-test")
results = []
errors = []
def download_worker(idx):
try:
from httpx import Client
base_url = os.environ.get("ORCHARD_TEST_URL", "http://localhost:8080")
with Client(base_url=base_url, timeout=60.0) as client:
response = client.get(
f"/api/v1/project/{project}/{package}/+/download-test",
params={"mode": "proxy"},
)
if response.status_code == 200:
results.append((idx, response.content))
else:
errors.append(f"Worker {idx}: Status {response.status_code}")
except Exception as e:
errors.append(f"Worker {idx}: {str(e)}")
with ThreadPoolExecutor(max_workers=2) as executor:
futures = [executor.submit(download_worker, i) for i in range(2)]
for future in as_completed(futures):
pass
assert len(errors) == 0, f"Errors: {errors}"
assert len(results) == 2
# All downloads should match original
for idx, downloaded in results:
assert downloaded == content
@pytest.mark.integration
@pytest.mark.concurrent
def test_5_concurrent_downloads_same_artifact(self, integration_client, test_package):
"""Test 5 concurrent downloads of same artifact."""
project, package = test_package
content, expected_hash = generate_content_with_hash(4096, seed=500)
upload_test_file(integration_client, project, package, content, tag="download5-test")
num_downloads = 5
results = []
errors = []
def download_worker(idx):
try:
from httpx import Client
base_url = os.environ.get("ORCHARD_TEST_URL", "http://localhost:8080")
with Client(base_url=base_url, timeout=60.0) as client:
response = client.get(
f"/api/v1/project/{project}/{package}/+/download5-test",
params={"mode": "proxy"},
)
if response.status_code == 200:
results.append((idx, response.content))
else:
errors.append(f"Worker {idx}: Status {response.status_code}")
except Exception as e:
errors.append(f"Worker {idx}: {str(e)}")
with ThreadPoolExecutor(max_workers=num_downloads) as executor:
futures = [executor.submit(download_worker, i) for i in range(num_downloads)]
for future in as_completed(futures):
pass
assert len(errors) == 0, f"Errors: {errors}"
assert len(results) == num_downloads
for idx, downloaded in results:
assert downloaded == content
@pytest.mark.integration
@pytest.mark.concurrent
def test_10_concurrent_downloads_same_artifact(self, integration_client, test_package):
"""Test 10 concurrent downloads of same artifact."""
project, package = test_package
content, expected_hash = generate_content_with_hash(8192, seed=600)
upload_test_file(integration_client, project, package, content, tag="download10-test")
num_downloads = 10
results = []
errors = []
def download_worker(idx):
try:
from httpx import Client
base_url = os.environ.get("ORCHARD_TEST_URL", "http://localhost:8080")
with Client(base_url=base_url, timeout=60.0) as client:
response = client.get(
f"/api/v1/project/{project}/{package}/+/download10-test",
params={"mode": "proxy"},
)
if response.status_code == 200:
results.append((idx, response.content))
else:
errors.append(f"Worker {idx}: Status {response.status_code}")
except Exception as e:
errors.append(f"Worker {idx}: {str(e)}")
with ThreadPoolExecutor(max_workers=num_downloads) as executor:
futures = [executor.submit(download_worker, i) for i in range(num_downloads)]
for future in as_completed(futures):
pass
assert len(errors) == 0, f"Errors: {errors}"
assert len(results) == num_downloads
for idx, downloaded in results:
assert downloaded == content
@pytest.mark.integration
@pytest.mark.concurrent
def test_concurrent_downloads_different_artifacts(self, integration_client, test_package):
"""Test concurrent downloads of different artifacts."""
project, package = test_package
# Upload multiple files
num_files = 5
uploads = []
for i in range(num_files):
content, expected_hash = generate_content_with_hash(1024, seed=700 + i)
upload_test_file(
integration_client, project, package, content,
tag=f"multi-download-{i}"
)
uploads.append((f"multi-download-{i}", content))
results = []
errors = []
def download_worker(tag, expected_content):
try:
from httpx import Client
base_url = os.environ.get("ORCHARD_TEST_URL", "http://localhost:8080")
with Client(base_url=base_url, timeout=60.0) as client:
response = client.get(
f"/api/v1/project/{project}/{package}/+/{tag}",
params={"mode": "proxy"},
)
if response.status_code == 200:
results.append((tag, response.content, expected_content))
else:
errors.append(f"Tag {tag}: Status {response.status_code}")
except Exception as e:
errors.append(f"Tag {tag}: {str(e)}")
with ThreadPoolExecutor(max_workers=num_files) as executor:
futures = [
executor.submit(download_worker, tag, content)
for tag, content in uploads
]
for future in as_completed(futures):
pass
assert len(errors) == 0, f"Errors: {errors}"
assert len(results) == num_files
for tag, downloaded, expected in results:
assert downloaded == expected, f"Content mismatch for {tag}"
class TestMixedConcurrentOperations:
"""Tests for mixed concurrent upload and download operations."""
@pytest.mark.integration
@pytest.mark.concurrent
def test_upload_while_download_in_progress(self, integration_client, test_package):
"""Test uploading while a download is in progress."""
project, package = test_package
api_key = get_api_key(integration_client)
assert api_key, "Failed to create API key"
# Upload initial content
content1, hash1 = generate_content_with_hash(10240, seed=800) # 10KB
upload_test_file(integration_client, project, package, content1, tag="initial")
# New content for upload during download
content2, hash2 = generate_content_with_hash(10240, seed=801)
results = {"downloads": [], "uploads": []}
errors = []
def download_worker():
try:
from httpx import Client
base_url = os.environ.get("ORCHARD_TEST_URL", "http://localhost:8080")
with Client(base_url=base_url, timeout=60.0) as client:
response = client.get(
f"/api/v1/project/{project}/{package}/+/initial",
params={"mode": "proxy"},
)
if response.status_code == 200:
results["downloads"].append(response.content)
else:
errors.append(f"Download: Status {response.status_code}")
except Exception as e:
errors.append(f"Download: {str(e)}")
def upload_worker():
try:
from httpx import Client
base_url = os.environ.get("ORCHARD_TEST_URL", "http://localhost:8080")
with Client(base_url=base_url, timeout=60.0) as client:
files = {
"file": ("new.bin", io.BytesIO(content2), "application/octet-stream")
}
response = client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
data={"tag": "during-download"},
headers={"Authorization": f"Bearer {api_key}"},
)
if response.status_code == 200:
results["uploads"].append(response.json())
else:
errors.append(f"Upload: Status {response.status_code}")
except Exception as e:
errors.append(f"Upload: {str(e)}")
with ThreadPoolExecutor(max_workers=2) as executor:
futures = [
executor.submit(download_worker),
executor.submit(upload_worker),
]
for future in as_completed(futures):
pass
assert len(errors) == 0, f"Errors: {errors}"
assert len(results["downloads"]) == 1
assert len(results["uploads"]) == 1
# Verify download got correct content
assert results["downloads"][0] == content1
# Verify upload succeeded
assert results["uploads"][0]["artifact_id"] == hash2
@pytest.mark.integration
@pytest.mark.concurrent
def test_multiple_uploads_and_downloads_simultaneously(self, integration_client, test_package):
"""Test multiple uploads and downloads running simultaneously."""
project, package = test_package
api_key = get_api_key(integration_client)
assert api_key, "Failed to create API key"
# Pre-upload some files for downloading
existing_files = []
for i in range(3):
content, hash = generate_content_with_hash(2048, seed=900 + i)
upload_test_file(integration_client, project, package, content, tag=f"existing-{i}")
existing_files.append((f"existing-{i}", content))
# New files for uploading
new_files = [
generate_content_with_hash(2048, seed=910 + i) for i in range(3)
]
results = {"downloads": [], "uploads": []}
errors = []
def download_worker(tag, expected):
try:
from httpx import Client
base_url = os.environ.get("ORCHARD_TEST_URL", "http://localhost:8080")
with Client(base_url=base_url, timeout=60.0) as client:
response = client.get(
f"/api/v1/project/{project}/{package}/+/{tag}",
params={"mode": "proxy"},
)
if response.status_code == 200:
results["downloads"].append((tag, response.content, expected))
else:
errors.append(f"Download {tag}: Status {response.status_code}")
except Exception as e:
errors.append(f"Download {tag}: {str(e)}")
def upload_worker(idx, content, expected_hash):
try:
from httpx import Client
base_url = os.environ.get("ORCHARD_TEST_URL", "http://localhost:8080")
with Client(base_url=base_url, timeout=60.0) as client:
files = {
"file": (f"new-{idx}.bin", io.BytesIO(content), "application/octet-stream")
}
response = client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
data={"tag": f"new-{idx}"},
headers={"Authorization": f"Bearer {api_key}"},
)
if response.status_code == 200:
results["uploads"].append((idx, response.json(), expected_hash))
else:
errors.append(f"Upload {idx}: Status {response.status_code}")
except Exception as e:
errors.append(f"Upload {idx}: {str(e)}")
with ThreadPoolExecutor(max_workers=6) as executor:
futures = []
# Submit downloads
for tag, content in existing_files:
futures.append(executor.submit(download_worker, tag, content))
# Submit uploads
for i, (content, hash) in enumerate(new_files):
futures.append(executor.submit(upload_worker, i, content, hash))
for future in as_completed(futures):
pass
assert len(errors) == 0, f"Errors: {errors}"
assert len(results["downloads"]) == 3
assert len(results["uploads"]) == 3
# Verify downloads
for tag, downloaded, expected in results["downloads"]:
assert downloaded == expected, f"Download mismatch for {tag}"
# Verify uploads
for idx, result, expected_hash in results["uploads"]:
assert result["artifact_id"] == expected_hash
@pytest.mark.integration
@pytest.mark.concurrent
def test_no_data_corruption_under_concurrency(self, integration_client, test_package):
"""Test that no data corruption occurs under concurrent operations."""
project, package = test_package
api_key = get_api_key(integration_client)
assert api_key, "Failed to create API key"
# Create content with recognizable patterns
num_files = 5
files_data = []
for i in range(num_files):
# Each file has unique repeating pattern for easy corruption detection
pattern = bytes([i] * 256)
content = pattern * 40 # 10KB each
hash = compute_sha256(content)
files_data.append((content, hash))
results = []
errors = []
def upload_and_verify(idx, content, expected_hash):
try:
from httpx import Client
base_url = os.environ.get("ORCHARD_TEST_URL", "http://localhost:8080")
with Client(base_url=base_url, timeout=60.0) as client:
# Upload
files = {
"file": (f"pattern-{idx}.bin", io.BytesIO(content), "application/octet-stream")
}
upload_resp = client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
data={"tag": f"pattern-{idx}"},
headers={"Authorization": f"Bearer {api_key}"},
)
if upload_resp.status_code != 200:
errors.append(f"Upload {idx}: Status {upload_resp.status_code}")
return
upload_result = upload_resp.json()
if upload_result["artifact_id"] != expected_hash:
errors.append(f"Upload {idx}: Hash mismatch")
return
# Immediately download and verify
download_resp = client.get(
f"/api/v1/project/{project}/{package}/+/pattern-{idx}",
params={"mode": "proxy"},
)
if download_resp.status_code != 200:
errors.append(f"Download {idx}: Status {download_resp.status_code}")
return
if download_resp.content != content:
errors.append(f"Worker {idx}: DATA CORRUPTION DETECTED")
return
# Verify the downloaded content hash
downloaded_hash = compute_sha256(download_resp.content)
if downloaded_hash != expected_hash:
errors.append(f"Worker {idx}: Hash verification failed")
return
results.append(idx)
except Exception as e:
errors.append(f"Worker {idx}: {str(e)}")
with ThreadPoolExecutor(max_workers=num_files) as executor:
futures = [
executor.submit(upload_and_verify, i, content, hash)
for i, (content, hash) in enumerate(files_data)
]
for future in as_completed(futures):
pass
assert len(errors) == 0, f"Errors: {errors}"
assert len(results) == num_files

View File

@@ -0,0 +1,322 @@
"""
Integration tests for error handling in upload and download operations.
Tests cover:
- Timeout handling
- Invalid request handling
- Resource cleanup on failures
- Graceful error responses
"""
import pytest
import io
import time
from tests.factories import (
compute_sha256,
upload_test_file,
generate_content_with_hash,
)
class TestUploadErrorHandling:
"""Tests for upload error handling."""
@pytest.mark.integration
def test_upload_to_nonexistent_project_returns_404(
self, integration_client, unique_test_id
):
"""Test upload to nonexistent project returns 404."""
content = b"test content for nonexistent project"
files = {"file": ("test.bin", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/nonexistent-project-{unique_test_id}/nonexistent-pkg/upload",
files=files,
)
assert response.status_code == 404
@pytest.mark.integration
def test_upload_to_nonexistent_package_returns_404(
self, integration_client, test_project, unique_test_id
):
"""Test upload to nonexistent package returns 404."""
content = b"test content for nonexistent package"
files = {"file": ("test.bin", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{test_project}/nonexistent-package-{unique_test_id}/upload",
files=files,
)
assert response.status_code == 404
@pytest.mark.integration
def test_upload_empty_file_rejected(self, integration_client, test_package):
"""Test empty file upload is rejected."""
project, package = test_package
files = {"file": ("empty.bin", io.BytesIO(b""), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
)
assert response.status_code in [400, 422]
@pytest.mark.integration
def test_upload_missing_file_returns_422(self, integration_client, test_package):
"""Test upload without file field returns 422."""
project, package = test_package
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
data={"tag": "no-file-provided"},
)
assert response.status_code == 422
@pytest.mark.integration
def test_upload_invalid_checksum_format_returns_400(
self, integration_client, test_package
):
"""Test upload with invalid checksum format returns 400."""
project, package = test_package
content = b"checksum format test"
files = {"file": ("test.bin", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
headers={"X-Checksum-SHA256": "invalid-hash-format"},
)
assert response.status_code == 400
@pytest.mark.integration
def test_upload_checksum_mismatch_returns_422(
self, integration_client, test_package
):
"""Test upload with mismatched checksum returns 422."""
project, package = test_package
content = b"checksum mismatch test"
wrong_hash = "0" * 64 # Valid format but wrong hash
files = {"file": ("test.bin", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
headers={"X-Checksum-SHA256": wrong_hash},
)
assert response.status_code == 422
@pytest.mark.integration
def test_upload_with_correct_checksum_succeeds(
self, integration_client, test_package
):
"""Test upload with correct checksum succeeds."""
project, package = test_package
content = b"correct checksum test"
correct_hash = compute_sha256(content)
files = {"file": ("test.bin", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
headers={"X-Checksum-SHA256": correct_hash},
)
assert response.status_code == 200
assert response.json()["artifact_id"] == correct_hash
class TestDownloadErrorHandling:
"""Tests for download error handling."""
@pytest.mark.integration
def test_download_nonexistent_tag_returns_404(
self, integration_client, test_package
):
"""Test download of nonexistent tag returns 404."""
project, package = test_package
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/nonexistent-tag-xyz"
)
assert response.status_code == 404
@pytest.mark.integration
def test_download_nonexistent_artifact_returns_404(
self, integration_client, test_package
):
"""Test download of nonexistent artifact ID returns 404."""
project, package = test_package
fake_hash = "a" * 64
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/artifact:{fake_hash}"
)
assert response.status_code == 404
@pytest.mark.integration
def test_download_invalid_artifact_id_format(
self, integration_client, test_package
):
"""Test download with invalid artifact ID format."""
project, package = test_package
# Too short
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/artifact:abc123"
)
assert response.status_code == 404
@pytest.mark.integration
def test_download_from_nonexistent_project_returns_404(
self, integration_client, unique_test_id
):
"""Test download from nonexistent project returns 404."""
response = integration_client.get(
f"/api/v1/project/nonexistent-{unique_test_id}/pkg/+/tag"
)
assert response.status_code == 404
@pytest.mark.integration
def test_download_from_nonexistent_package_returns_404(
self, integration_client, test_project, unique_test_id
):
"""Test download from nonexistent package returns 404."""
response = integration_client.get(
f"/api/v1/project/{test_project}/nonexistent-{unique_test_id}/+/tag"
)
assert response.status_code == 404
class TestTimeoutBehavior:
"""Tests for timeout behavior (integration level)."""
@pytest.mark.integration
@pytest.mark.slow
def test_large_upload_completes_within_reasonable_time(
self, integration_client, test_package, sized_content
):
"""Test that a 10MB upload completes within reasonable time."""
project, package = test_package
content, expected_hash = sized_content(10 * 1024 * 1024, seed=999) # 10MB
start_time = time.time()
result = upload_test_file(
integration_client, project, package, content, tag="timeout-test"
)
elapsed = time.time() - start_time
assert result["artifact_id"] == expected_hash
# Should complete within 60 seconds for 10MB on local docker
assert elapsed < 60, f"Upload took too long: {elapsed:.2f}s"
@pytest.mark.integration
@pytest.mark.slow
def test_large_download_completes_within_reasonable_time(
self, integration_client, test_package, sized_content
):
"""Test that a 10MB download completes within reasonable time."""
project, package = test_package
content, expected_hash = sized_content(10 * 1024 * 1024, seed=998) # 10MB
# First upload
upload_test_file(
integration_client, project, package, content, tag="download-timeout-test"
)
# Then download and time it
start_time = time.time()
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/download-timeout-test",
params={"mode": "proxy"},
)
elapsed = time.time() - start_time
assert response.status_code == 200
assert len(response.content) == len(content)
# Should complete within 60 seconds for 10MB on local docker
assert elapsed < 60, f"Download took too long: {elapsed:.2f}s"
class TestResourceCleanup:
"""Tests for proper resource cleanup on failures.
Note: More comprehensive cleanup tests are in test_upload_download_api.py
(TestUploadFailureCleanup class) including S3 object cleanup verification.
"""
@pytest.mark.integration
def test_checksum_mismatch_no_orphaned_artifact(
self, integration_client, test_package, unique_test_id
):
"""Test checksum mismatch doesn't leave orphaned artifact."""
project, package = test_package
# Use unique content to ensure artifact doesn't exist from prior tests
content = f"checksum mismatch orphan test {unique_test_id}".encode()
wrong_hash = "0" * 64
actual_hash = compute_sha256(content)
# Verify artifact doesn't exist before test
pre_check = integration_client.get(f"/api/v1/artifact/{actual_hash}")
assert pre_check.status_code == 404, "Artifact should not exist before test"
files = {"file": ("test.bin", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
headers={"X-Checksum-SHA256": wrong_hash},
)
assert response.status_code == 422
# Verify no artifact was created with either hash
response1 = integration_client.get(f"/api/v1/artifact/{wrong_hash}")
response2 = integration_client.get(f"/api/v1/artifact/{actual_hash}")
assert response1.status_code == 404
assert response2.status_code == 404
class TestGracefulErrorResponses:
"""Tests for graceful and informative error responses."""
@pytest.mark.integration
def test_404_response_has_detail_message(
self, integration_client, test_package
):
"""Test 404 responses include a detail message."""
project, package = test_package
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/nonexistent-tag"
)
assert response.status_code == 404
data = response.json()
assert "detail" in data
assert len(data["detail"]) > 0
@pytest.mark.integration
def test_422_response_has_detail_message(self, integration_client, test_package):
"""Test 422 responses include a detail message."""
project, package = test_package
# Upload with mismatched checksum
content = b"detail message test"
wrong_hash = "0" * 64
files = {"file": ("test.bin", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
headers={"X-Checksum-SHA256": wrong_hash},
)
assert response.status_code == 422
data = response.json()
assert "detail" in data
@pytest.mark.integration
def test_error_response_is_json(self, integration_client, unique_test_id):
"""Test error responses are valid JSON."""
response = integration_client.get(
f"/api/v1/project/nonexistent-{unique_test_id}/pkg/+/tag"
)
assert response.status_code == 404
# Should not raise exception - valid JSON
data = response.json()
assert isinstance(data, dict)

View File

@@ -0,0 +1,768 @@
"""
Integration tests for artifact integrity verification.
Tests cover:
- Round-trip verification (upload -> download -> verify hash)
- Consistency check endpoint
- Header-based verification
- Integrity verification across file sizes
- Client-side verification workflow
"""
import pytest
import io
import hashlib
from tests.factories import (
compute_sha256,
upload_test_file,
generate_content_with_hash,
s3_object_exists,
get_s3_client,
get_s3_bucket,
)
from tests.conftest import (
SIZE_1KB,
SIZE_10KB,
SIZE_100KB,
SIZE_1MB,
SIZE_10MB,
)
class TestRoundTripVerification:
"""Tests for complete round-trip integrity verification."""
@pytest.mark.integration
def test_upload_download_hash_matches(self, integration_client, test_package):
"""Test that upload -> download round trip preserves content integrity."""
project, package = test_package
content = b"Round trip integrity test content"
expected_hash = compute_sha256(content)
# Upload and capture returned hash
result = upload_test_file(
integration_client, project, package, content, tag="roundtrip"
)
uploaded_hash = result["artifact_id"]
# Verify upload returned correct hash
assert uploaded_hash == expected_hash
# Download artifact
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/roundtrip",
params={"mode": "proxy"},
)
assert response.status_code == 200
# Compute hash of downloaded content
downloaded_hash = compute_sha256(response.content)
# All three hashes should match
assert downloaded_hash == expected_hash
assert downloaded_hash == uploaded_hash
@pytest.mark.integration
def test_upload_response_contains_hash(self, integration_client, test_package):
"""Test upload response contains artifact_id which is the SHA256 hash."""
project, package = test_package
content = b"Upload response hash test"
expected_hash = compute_sha256(content)
result = upload_test_file(integration_client, project, package, content)
assert "artifact_id" in result
assert result["artifact_id"] == expected_hash
assert len(result["artifact_id"]) == 64
assert all(c in "0123456789abcdef" for c in result["artifact_id"])
@pytest.mark.integration
def test_download_header_matches_artifact_id(self, integration_client, test_package):
"""Test X-Checksum-SHA256 header matches artifact ID."""
project, package = test_package
content = b"Header verification test"
expected_hash = compute_sha256(content)
upload_test_file(
integration_client, project, package, content, tag="header-check"
)
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/header-check",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert response.headers.get("X-Checksum-SHA256") == expected_hash
@pytest.mark.integration
def test_etag_matches_artifact_id(self, integration_client, test_package):
"""Test ETag header matches artifact ID."""
project, package = test_package
content = b"ETag verification test"
expected_hash = compute_sha256(content)
upload_test_file(
integration_client, project, package, content, tag="etag-check"
)
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/etag-check",
params={"mode": "proxy"},
)
assert response.status_code == 200
etag = response.headers.get("ETag", "").strip('"')
assert etag == expected_hash
@pytest.mark.integration
def test_artifact_endpoint_returns_correct_hash(self, integration_client, test_package):
"""Test artifact endpoint returns correct hash/ID."""
project, package = test_package
content = b"Artifact endpoint hash test"
expected_hash = compute_sha256(content)
upload_test_file(integration_client, project, package, content)
# Query artifact directly
response = integration_client.get(f"/api/v1/artifact/{expected_hash}")
assert response.status_code == 200
data = response.json()
assert data["id"] == expected_hash
assert data.get("sha256") == expected_hash
class TestClientSideVerificationWorkflow:
"""Tests for client-side verification workflow."""
@pytest.mark.integration
def test_client_can_verify_before_upload(self, integration_client, test_package):
"""Test client can compute hash before upload and verify response matches."""
project, package = test_package
content = b"Client pre-upload verification test"
# Client computes hash locally before upload
client_hash = compute_sha256(content)
# Upload
result = upload_test_file(integration_client, project, package, content)
# Client verifies server returned the same hash
assert result["artifact_id"] == client_hash
@pytest.mark.integration
def test_client_can_provide_checksum_header(self, integration_client, test_package):
"""Test client can provide X-Checksum-SHA256 header for verification."""
project, package = test_package
content = b"Client checksum header test"
client_hash = compute_sha256(content)
files = {"file": ("test.bin", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
headers={"X-Checksum-SHA256": client_hash},
)
assert response.status_code == 200
assert response.json()["artifact_id"] == client_hash
@pytest.mark.integration
def test_checksum_mismatch_rejected(self, integration_client, test_package):
"""Test upload with wrong client checksum is rejected."""
project, package = test_package
content = b"Checksum mismatch test"
wrong_hash = "0" * 64
files = {"file": ("test.bin", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
headers={"X-Checksum-SHA256": wrong_hash},
)
assert response.status_code == 422
@pytest.mark.integration
def test_client_can_verify_after_download(self, integration_client, test_package):
"""Test client can verify downloaded content matches header hash."""
project, package = test_package
content = b"Client post-download verification"
upload_test_file(
integration_client, project, package, content, tag="verify-after"
)
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/verify-after",
params={"mode": "proxy"},
)
assert response.status_code == 200
# Client gets hash from header
header_hash = response.headers.get("X-Checksum-SHA256")
# Client computes hash of downloaded content
downloaded_hash = compute_sha256(response.content)
# Client verifies they match
assert downloaded_hash == header_hash
class TestIntegritySizeVariants:
"""Tests for integrity verification across different file sizes."""
@pytest.mark.integration
def test_integrity_1kb(self, integration_client, test_package, sized_content):
"""Test integrity verification for 1KB file."""
project, package = test_package
content, expected_hash = sized_content(SIZE_1KB, seed=100)
result = upload_test_file(
integration_client, project, package, content, tag="int-1kb"
)
assert result["artifact_id"] == expected_hash
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/int-1kb",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert compute_sha256(response.content) == expected_hash
assert response.headers.get("X-Checksum-SHA256") == expected_hash
@pytest.mark.integration
def test_integrity_100kb(self, integration_client, test_package, sized_content):
"""Test integrity verification for 100KB file."""
project, package = test_package
content, expected_hash = sized_content(SIZE_100KB, seed=101)
result = upload_test_file(
integration_client, project, package, content, tag="int-100kb"
)
assert result["artifact_id"] == expected_hash
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/int-100kb",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert compute_sha256(response.content) == expected_hash
assert response.headers.get("X-Checksum-SHA256") == expected_hash
@pytest.mark.integration
def test_integrity_1mb(self, integration_client, test_package, sized_content):
"""Test integrity verification for 1MB file."""
project, package = test_package
content, expected_hash = sized_content(SIZE_1MB, seed=102)
result = upload_test_file(
integration_client, project, package, content, tag="int-1mb"
)
assert result["artifact_id"] == expected_hash
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/int-1mb",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert compute_sha256(response.content) == expected_hash
assert response.headers.get("X-Checksum-SHA256") == expected_hash
@pytest.mark.integration
@pytest.mark.slow
def test_integrity_10mb(self, integration_client, test_package, sized_content):
"""Test integrity verification for 10MB file."""
project, package = test_package
content, expected_hash = sized_content(SIZE_10MB, seed=103)
result = upload_test_file(
integration_client, project, package, content, tag="int-10mb"
)
assert result["artifact_id"] == expected_hash
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/int-10mb",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert compute_sha256(response.content) == expected_hash
assert response.headers.get("X-Checksum-SHA256") == expected_hash
class TestConsistencyCheck:
"""Tests for the admin consistency check endpoint."""
@pytest.mark.integration
def test_consistency_check_returns_200(self, integration_client):
"""Test consistency check endpoint returns 200."""
response = integration_client.get("/api/v1/admin/consistency-check")
assert response.status_code == 200
@pytest.mark.integration
def test_consistency_check_response_format(self, integration_client):
"""Test consistency check returns expected response format."""
response = integration_client.get("/api/v1/admin/consistency-check")
assert response.status_code == 200
data = response.json()
# Check expected fields
assert "total_artifacts_checked" in data
assert "orphaned_s3_objects" in data
assert "missing_s3_objects" in data
assert "size_mismatches" in data
assert "healthy" in data
assert "orphaned_s3_keys" in data
assert "missing_s3_keys" in data
assert "size_mismatch_artifacts" in data
# Verify types
assert isinstance(data["total_artifacts_checked"], int)
assert isinstance(data["orphaned_s3_objects"], int)
assert isinstance(data["missing_s3_objects"], int)
assert isinstance(data["size_mismatches"], int)
assert isinstance(data["healthy"], bool)
assert isinstance(data["orphaned_s3_keys"], list)
assert isinstance(data["missing_s3_keys"], list)
assert isinstance(data["size_mismatch_artifacts"], list)
@pytest.mark.integration
def test_consistency_check_after_upload(self, integration_client, test_package):
"""Test consistency check passes after valid upload."""
project, package = test_package
content = b"Consistency check test content"
# Upload artifact
upload_test_file(integration_client, project, package, content)
# Run consistency check
response = integration_client.get("/api/v1/admin/consistency-check")
assert response.status_code == 200
data = response.json()
# Verify check ran and no issues
assert data["total_artifacts_checked"] >= 1
assert data["healthy"] is True
@pytest.mark.integration
def test_consistency_check_limit_parameter(self, integration_client):
"""Test consistency check respects limit parameter."""
response = integration_client.get(
"/api/v1/admin/consistency-check",
params={"limit": 10}
)
assert response.status_code == 200
data = response.json()
# Lists should not exceed limit
assert len(data["orphaned_s3_keys"]) <= 10
assert len(data["missing_s3_keys"]) <= 10
assert len(data["size_mismatch_artifacts"]) <= 10
class TestDigestHeader:
"""Tests for RFC 3230 Digest header."""
@pytest.mark.integration
def test_download_includes_digest_header(self, integration_client, test_package):
"""Test download includes Digest header in RFC 3230 format."""
project, package = test_package
content = b"Digest header test"
expected_hash = compute_sha256(content)
upload_test_file(
integration_client, project, package, content, tag="digest-test"
)
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/digest-test",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert "Digest" in response.headers
# Verify Digest format (sha-256=base64hash)
digest = response.headers["Digest"]
assert digest.startswith("sha-256=")
@pytest.mark.integration
def test_digest_header_base64_valid(self, integration_client, test_package):
"""Test Digest header contains valid base64 encoding."""
import base64
project, package = test_package
content = b"Digest base64 test"
expected_hash = compute_sha256(content)
upload_test_file(
integration_client, project, package, content, tag="digest-b64"
)
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/digest-b64",
params={"mode": "proxy"},
)
assert response.status_code == 200
digest = response.headers["Digest"]
base64_part = digest.split("=", 1)[1]
# Should be valid base64
try:
decoded = base64.b64decode(base64_part)
assert len(decoded) == 32 # SHA256 is 32 bytes
except Exception as e:
pytest.fail(f"Invalid base64 in Digest header: {e}")
class TestVerificationModes:
"""Tests for download verification modes."""
@pytest.mark.integration
def test_pre_verification_mode(self, integration_client, test_package):
"""Test pre-verification mode verifies before streaming."""
project, package = test_package
content = b"Pre-verification mode test"
upload_test_file(
integration_client, project, package, content, tag="pre-verify"
)
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/pre-verify",
params={"mode": "proxy", "verify": "true", "verify_mode": "pre"},
)
assert response.status_code == 200
assert response.content == content
# X-Verified header should be true
assert response.headers.get("X-Verified") == "true"
@pytest.mark.integration
def test_stream_verification_mode(self, integration_client, test_package):
"""Test streaming verification mode."""
project, package = test_package
content = b"Stream verification mode test"
upload_test_file(
integration_client, project, package, content, tag="stream-verify"
)
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/stream-verify",
params={"mode": "proxy", "verify": "true", "verify_mode": "stream"},
)
assert response.status_code == 200
assert response.content == content
class TestArtifactIntegrityEndpoint:
"""Tests for artifact-specific integrity operations."""
@pytest.mark.integration
def test_artifact_size_matches(self, integration_client, test_package):
"""Test artifact endpoint returns correct size."""
project, package = test_package
content = b"Artifact size test content"
expected_size = len(content)
result = upload_test_file(integration_client, project, package, content)
artifact_id = result["artifact_id"]
response = integration_client.get(f"/api/v1/artifact/{artifact_id}")
assert response.status_code == 200
data = response.json()
assert data["size"] == expected_size
@pytest.mark.integration
def test_content_length_header_matches_size(self, integration_client, test_package):
"""Test Content-Length header matches artifact size."""
project, package = test_package
content = b"Content-Length header test"
expected_size = len(content)
upload_test_file(
integration_client, project, package, content, tag="content-len"
)
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/content-len",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert int(response.headers.get("Content-Length", 0)) == expected_size
assert len(response.content) == expected_size
@pytest.mark.requires_direct_s3
class TestCorruptionDetection:
"""Tests for detecting corrupted S3 objects.
These tests directly manipulate S3 objects to simulate corruption
and verify that the system can detect hash mismatches.
Note: These tests require direct S3/MinIO access and are skipped in CI
where S3 is not directly accessible from the test runner.
"""
@pytest.mark.integration
def test_detection_of_corrupted_content(self, integration_client, test_package):
"""Test that corrupted S3 content is detected via hash mismatch.
Uploads content, then directly modifies the S3 object, then
verifies that the downloaded content hash doesn't match.
"""
project, package = test_package
content = b"Original content for corruption test"
expected_hash = compute_sha256(content)
# Upload original content
result = upload_test_file(
integration_client, project, package, content, tag="corrupt-test"
)
assert result["artifact_id"] == expected_hash
# Get the S3 object and corrupt it
s3_client = get_s3_client()
bucket = get_s3_bucket()
s3_key = f"fruits/{expected_hash[:2]}/{expected_hash[2:4]}/{expected_hash}"
# Replace with corrupted content
corrupted_content = b"Corrupted content - different from original!"
s3_client.put_object(Bucket=bucket, Key=s3_key, Body=corrupted_content)
# Download via proxy (bypasses hash verification)
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/corrupt-test",
params={"mode": "proxy"},
)
assert response.status_code == 200
# Verify the downloaded content doesn't match original hash
downloaded_hash = compute_sha256(response.content)
assert downloaded_hash != expected_hash, "Corruption was not detected - hashes match"
assert response.content == corrupted_content
# The X-Checksum-SHA256 header should still show the original hash (from DB)
# but the actual content hash is different
header_hash = response.headers.get("X-Checksum-SHA256")
assert header_hash == expected_hash # Header shows expected hash
assert downloaded_hash != header_hash # But content is corrupted
# Restore original content for cleanup
s3_client.put_object(Bucket=bucket, Key=s3_key, Body=content)
@pytest.mark.integration
def test_detection_of_single_bit_flip(self, integration_client, test_package):
"""Test detection of a single bit flip in S3 object content."""
project, package = test_package
content = b"Content for single bit flip detection test"
expected_hash = compute_sha256(content)
result = upload_test_file(
integration_client, project, package, content, tag="bitflip-test"
)
assert result["artifact_id"] == expected_hash
# Get S3 object and flip a single bit
s3_client = get_s3_client()
bucket = get_s3_bucket()
s3_key = f"fruits/{expected_hash[:2]}/{expected_hash[2:4]}/{expected_hash}"
# Flip the first bit of the first byte
corrupted_content = bytearray(content)
corrupted_content[0] ^= 0x01
corrupted_content = bytes(corrupted_content)
s3_client.put_object(Bucket=bucket, Key=s3_key, Body=corrupted_content)
# Download and verify hash mismatch
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/bitflip-test",
params={"mode": "proxy"},
)
assert response.status_code == 200
downloaded_hash = compute_sha256(response.content)
assert downloaded_hash != expected_hash, "Single bit flip not detected"
# Restore original
s3_client.put_object(Bucket=bucket, Key=s3_key, Body=content)
@pytest.mark.integration
def test_detection_of_truncated_content(self, integration_client, test_package):
"""Test detection of truncated S3 object."""
project, package = test_package
content = b"This is content that will be truncated for testing purposes"
expected_hash = compute_sha256(content)
result = upload_test_file(
integration_client, project, package, content, tag="truncate-test"
)
assert result["artifact_id"] == expected_hash
# Get S3 object and truncate it
s3_client = get_s3_client()
bucket = get_s3_bucket()
s3_key = f"fruits/{expected_hash[:2]}/{expected_hash[2:4]}/{expected_hash}"
# Truncate to half the original size
truncated_content = content[: len(content) // 2]
s3_client.put_object(Bucket=bucket, Key=s3_key, Body=truncated_content)
# Download and verify hash mismatch
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/truncate-test",
params={"mode": "proxy"},
)
assert response.status_code == 200
downloaded_hash = compute_sha256(response.content)
assert downloaded_hash != expected_hash, "Truncation not detected"
assert len(response.content) < len(content), "Content was not truncated"
# Restore original
s3_client.put_object(Bucket=bucket, Key=s3_key, Body=content)
@pytest.mark.integration
def test_detection_of_appended_content(self, integration_client, test_package):
"""Test detection of content with extra bytes appended."""
project, package = test_package
content = b"Original content"
expected_hash = compute_sha256(content)
result = upload_test_file(
integration_client, project, package, content, tag="append-test"
)
assert result["artifact_id"] == expected_hash
# Get S3 object and append extra bytes
s3_client = get_s3_client()
bucket = get_s3_bucket()
s3_key = f"fruits/{expected_hash[:2]}/{expected_hash[2:4]}/{expected_hash}"
appended_content = content + b" - extra bytes appended"
s3_client.put_object(Bucket=bucket, Key=s3_key, Body=appended_content)
# Download and verify hash mismatch
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/append-test",
params={"mode": "proxy"},
)
assert response.status_code == 200
downloaded_hash = compute_sha256(response.content)
assert downloaded_hash != expected_hash, "Appended content not detected"
assert len(response.content) > len(content), "Content was not extended"
# Restore original
s3_client.put_object(Bucket=bucket, Key=s3_key, Body=content)
@pytest.mark.integration
def test_client_detects_hash_mismatch_post_download(
self, integration_client, test_package
):
"""Test that a client can detect hash mismatch after downloading corrupted content.
This simulates the full client verification workflow:
1. Download content
2. Get expected hash from header
3. Compute actual hash of content
4. Verify they match (or detect corruption)
"""
project, package = test_package
content = b"Content for client-side corruption detection"
expected_hash = compute_sha256(content)
result = upload_test_file(
integration_client, project, package, content, tag="client-detect"
)
# Corrupt the S3 object
s3_client = get_s3_client()
bucket = get_s3_bucket()
s3_key = f"fruits/{expected_hash[:2]}/{expected_hash[2:4]}/{expected_hash}"
corrupted = b"This is completely different content"
s3_client.put_object(Bucket=bucket, Key=s3_key, Body=corrupted)
# Simulate client download and verification
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/client-detect",
params={"mode": "proxy"},
)
assert response.status_code == 200
# Client gets expected hash from header
header_hash = response.headers.get("X-Checksum-SHA256")
# Client computes hash of downloaded content
actual_hash = compute_sha256(response.content)
# Client detects the mismatch
corruption_detected = actual_hash != header_hash
assert corruption_detected, "Client should detect hash mismatch"
# Restore original
s3_client.put_object(Bucket=bucket, Key=s3_key, Body=content)
@pytest.mark.integration
def test_consistency_check_detects_size_mismatch(
self, integration_client, test_package, unique_test_id
):
"""Test that consistency check detects size mismatches.
Uploads content, modifies S3 object size, then runs consistency check.
"""
project, package = test_package
content = b"Content for size mismatch consistency check test " + unique_test_id.encode()
expected_hash = compute_sha256(content)
result = upload_test_file(
integration_client, project, package, content, tag="size-mismatch"
)
# Modify S3 object to have different size
s3_client = get_s3_client()
bucket = get_s3_bucket()
s3_key = f"fruits/{expected_hash[:2]}/{expected_hash[2:4]}/{expected_hash}"
different_size_content = content + b"extra extra extra"
s3_client.put_object(Bucket=bucket, Key=s3_key, Body=different_size_content)
# Run consistency check
response = integration_client.get("/api/v1/admin/consistency-check")
assert response.status_code == 200
data = response.json()
# Should detect the size mismatch
assert data["size_mismatches"] >= 1 or len(data["size_mismatch_artifacts"]) >= 1
# Restore original
s3_client.put_object(Bucket=bucket, Key=s3_key, Body=content)
@pytest.mark.integration
def test_consistency_check_detects_missing_s3_object(
self, integration_client, test_package, unique_test_id
):
"""Test that consistency check detects missing S3 objects.
Uploads content, deletes S3 object, then runs consistency check.
"""
project, package = test_package
content = b"Content for missing S3 object test " + unique_test_id.encode()
expected_hash = compute_sha256(content)
result = upload_test_file(
integration_client, project, package, content, tag="missing-s3"
)
# Delete the S3 object
s3_client = get_s3_client()
bucket = get_s3_bucket()
s3_key = f"fruits/{expected_hash[:2]}/{expected_hash[2:4]}/{expected_hash}"
s3_client.delete_object(Bucket=bucket, Key=s3_key)
# Run consistency check
response = integration_client.get("/api/v1/admin/consistency-check")
assert response.status_code == 200
data = response.json()
# Should detect the missing S3 object
assert data["missing_s3_objects"] >= 1 or len(data["missing_s3_keys"]) >= 1
# Restore the object for cleanup
s3_client.put_object(Bucket=bucket, Key=s3_key, Body=content)

View File

@@ -0,0 +1,552 @@
"""
Integration tests for large file upload functionality.
Tests cover:
- Large file uploads (100MB, 1GB)
- Multipart upload behavior
- Upload metrics (duration, throughput)
- Memory efficiency during uploads
- Upload progress tracking
Note: Large tests are marked with @pytest.mark.slow and will be skipped
by default. Run with `pytest --run-slow` to include them.
"""
import os
import pytest
import io
import time
from tests.factories import (
compute_sha256,
upload_test_file,
s3_object_exists,
)
from tests.conftest import (
SIZE_1KB,
SIZE_100KB,
SIZE_1MB,
SIZE_10MB,
SIZE_100MB,
SIZE_1GB,
)
class TestUploadMetrics:
"""Tests for upload duration and throughput metrics."""
@pytest.mark.integration
def test_upload_response_includes_duration_ms(self, integration_client, test_package):
"""Test upload response includes duration_ms field."""
project, package = test_package
content = b"duration test content"
result = upload_test_file(
integration_client, project, package, content, tag="duration-test"
)
assert "duration_ms" in result
assert result["duration_ms"] is not None
assert result["duration_ms"] >= 0
@pytest.mark.integration
def test_upload_response_includes_throughput(self, integration_client, test_package):
"""Test upload response includes throughput_mbps field."""
project, package = test_package
content = b"throughput test content"
result = upload_test_file(
integration_client, project, package, content, tag="throughput-test"
)
assert "throughput_mbps" in result
# For small files throughput may be very high or None
# Just verify the field exists
@pytest.mark.integration
def test_upload_duration_reasonable(
self, integration_client, test_package, sized_content
):
"""Test upload duration is reasonable for file size."""
project, package = test_package
content, _ = sized_content(SIZE_1MB, seed=100)
start = time.time()
result = upload_test_file(
integration_client, project, package, content, tag="duration-check"
)
actual_duration = (time.time() - start) * 1000 # ms
# Reported duration should be close to actual
assert result["duration_ms"] is not None
# Allow some variance (network overhead)
assert result["duration_ms"] <= actual_duration + 1000 # Within 1s
class TestLargeFileUploads:
"""Tests for large file uploads using multipart."""
@pytest.mark.integration
def test_upload_10mb_file(self, integration_client, test_package, sized_content):
"""Test uploading a 10MB file."""
project, package = test_package
content, expected_hash = sized_content(SIZE_10MB, seed=200)
result = upload_test_file(
integration_client, project, package, content, tag="large-10mb"
)
assert result["artifact_id"] == expected_hash
assert result["size"] == SIZE_10MB
assert result["duration_ms"] is not None
assert result["throughput_mbps"] is not None
@pytest.mark.integration
@pytest.mark.slow
@pytest.mark.requires_direct_s3
def test_upload_100mb_file(self, integration_client, test_package, sized_content):
"""Test uploading a 100MB file (triggers multipart upload)."""
project, package = test_package
content, expected_hash = sized_content(SIZE_100MB, seed=300)
result = upload_test_file(
integration_client, project, package, content, tag="large-100mb"
)
assert result["artifact_id"] == expected_hash
assert result["size"] == SIZE_100MB
# Verify S3 object exists
assert s3_object_exists(expected_hash)
@pytest.mark.integration
@pytest.mark.slow
@pytest.mark.large
def test_upload_1gb_file(self, integration_client, test_package, sized_content):
"""Test uploading a 1GB file."""
project, package = test_package
content, expected_hash = sized_content(SIZE_1GB, seed=400)
result = upload_test_file(
integration_client, project, package, content, tag="large-1gb"
)
assert result["artifact_id"] == expected_hash
assert result["size"] == SIZE_1GB
# Should have measurable throughput
assert result["throughput_mbps"] is not None
assert result["throughput_mbps"] > 0
@pytest.mark.integration
def test_large_file_deduplication(
self, integration_client, test_package, sized_content, unique_test_id
):
"""Test deduplication works for large files."""
project, package = test_package
# Use unique_test_id to ensure unique content per test run
seed = hash(unique_test_id) % 10000
content, expected_hash = sized_content(SIZE_10MB, seed=seed)
# First upload
result1 = upload_test_file(
integration_client, project, package, content, tag=f"dedup-{unique_test_id}-1"
)
# Note: may be True if previous test uploaded same content
first_dedupe = result1["deduplicated"]
# Second upload of same content
result2 = upload_test_file(
integration_client, project, package, content, tag=f"dedup-{unique_test_id}-2"
)
assert result2["artifact_id"] == expected_hash
# Second upload MUST be deduplicated
assert result2["deduplicated"] is True
class TestUploadProgress:
"""Tests for upload progress tracking endpoint."""
@pytest.mark.integration
def test_progress_endpoint_returns_not_found_for_invalid_id(
self, integration_client, test_package
):
"""Test progress endpoint returns not_found status for invalid upload ID."""
project, package = test_package
response = integration_client.get(
f"/api/v1/project/{project}/{package}/upload/invalid-upload-id/progress"
)
assert response.status_code == 200
data = response.json()
assert data["status"] == "not_found"
assert data["upload_id"] == "invalid-upload-id"
@pytest.mark.integration
def test_progress_endpoint_requires_valid_project(
self, integration_client, unique_test_id
):
"""Test progress endpoint validates project exists."""
response = integration_client.get(
f"/api/v1/project/nonexistent-{unique_test_id}/pkg/upload/upload-id/progress"
)
assert response.status_code == 404
@pytest.mark.integration
def test_progress_endpoint_requires_valid_package(
self, integration_client, test_project, unique_test_id
):
"""Test progress endpoint validates package exists."""
response = integration_client.get(
f"/api/v1/project/{test_project}/nonexistent-{unique_test_id}/upload/upload-id/progress"
)
assert response.status_code == 404
class TestResumableUploadProgress:
"""Tests for progress tracking during resumable uploads."""
@pytest.mark.integration
def test_resumable_upload_init_and_progress(
self, integration_client, test_package, sized_content
):
"""Test initializing resumable upload and checking progress."""
project, package = test_package
content, expected_hash = sized_content(SIZE_100KB, seed=600)
# Get API key for auth
api_key_response = integration_client.post(
"/api/v1/auth/keys",
json={"name": "progress-test-key"},
)
assert api_key_response.status_code == 200
api_key = api_key_response.json()["key"]
# Initialize resumable upload
init_response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload/init",
json={
"expected_hash": expected_hash,
"filename": "progress-test.bin",
"size": SIZE_100KB,
},
headers={"Authorization": f"Bearer {api_key}"},
)
assert init_response.status_code == 200
upload_id = init_response.json().get("upload_id")
if upload_id:
# Check initial progress
progress_response = integration_client.get(
f"/api/v1/project/{project}/{package}/upload/{upload_id}/progress",
headers={"Authorization": f"Bearer {api_key}"},
)
assert progress_response.status_code == 200
progress = progress_response.json()
assert progress["status"] == "in_progress"
assert progress["bytes_uploaded"] == 0
assert progress["bytes_total"] == SIZE_100KB
# Abort to clean up
integration_client.delete(
f"/api/v1/project/{project}/{package}/upload/{upload_id}",
headers={"Authorization": f"Bearer {api_key}"},
)
class TestUploadSizeLimits:
"""Tests for upload size limit enforcement."""
@pytest.mark.integration
def test_empty_file_rejected(self, integration_client, test_package):
"""Test empty files are rejected."""
project, package = test_package
files = {"file": ("empty.txt", io.BytesIO(b""), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
)
assert response.status_code in [400, 422]
@pytest.mark.integration
def test_minimum_size_accepted(self, integration_client, test_package):
"""Test 1-byte file is accepted."""
project, package = test_package
content = b"X"
result = upload_test_file(
integration_client, project, package, content, tag="min-size"
)
assert result["size"] == 1
@pytest.mark.integration
def test_content_length_header_used_in_response(self, integration_client, test_package):
"""Test that upload response size matches Content-Length."""
project, package = test_package
content = b"content length verification test"
result = upload_test_file(
integration_client, project, package, content, tag="content-length-test"
)
# Size in response should match actual content length
assert result["size"] == len(content)
class TestUploadErrorHandling:
"""Tests for upload error handling."""
@pytest.mark.integration
def test_upload_to_nonexistent_project_returns_404(
self, integration_client, unique_test_id
):
"""Test upload to nonexistent project returns 404."""
content = b"test content"
files = {"file": ("test.bin", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/nonexistent-{unique_test_id}/pkg/upload",
files=files,
)
assert response.status_code == 404
@pytest.mark.integration
def test_upload_to_nonexistent_package_returns_404(
self, integration_client, test_project, unique_test_id
):
"""Test upload to nonexistent package returns 404."""
content = b"test content"
files = {"file": ("test.bin", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{test_project}/nonexistent-{unique_test_id}/upload",
files=files,
)
assert response.status_code == 404
@pytest.mark.integration
def test_upload_without_file_returns_422(self, integration_client, test_package):
"""Test upload without file field returns 422."""
project, package = test_package
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
data={"tag": "no-file"},
)
assert response.status_code == 422
@pytest.mark.integration
def test_upload_with_invalid_checksum_rejected(
self, integration_client, test_package
):
"""Test upload with invalid checksum header format is rejected."""
project, package = test_package
content = b"checksum test"
files = {"file": ("test.bin", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
headers={"X-Checksum-SHA256": "invalid-checksum"},
)
assert response.status_code == 400
@pytest.mark.integration
def test_upload_with_mismatched_checksum_rejected(
self, integration_client, test_package
):
"""Test upload with wrong checksum is rejected."""
project, package = test_package
content = b"mismatch test"
wrong_hash = "0" * 64
files = {"file": ("test.bin", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
headers={"X-Checksum-SHA256": wrong_hash},
)
assert response.status_code == 422
assert "verification failed" in response.json().get("detail", "").lower()
class TestResumableUploadCancellation:
"""Tests for resumable upload cancellation."""
@pytest.mark.integration
def test_abort_resumable_upload(self, integration_client, test_package, sized_content):
"""Test aborting a resumable upload cleans up properly."""
project, package = test_package
content, expected_hash = sized_content(SIZE_100KB, seed=700)
# Get API key for auth
api_key_response = integration_client.post(
"/api/v1/auth/keys",
json={"name": "abort-test-key"},
)
assert api_key_response.status_code == 200
api_key = api_key_response.json()["key"]
# Initialize resumable upload
init_response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload/init",
json={
"expected_hash": expected_hash,
"filename": "abort-test.bin",
"size": SIZE_100KB,
},
headers={"Authorization": f"Bearer {api_key}"},
)
assert init_response.status_code == 200
upload_id = init_response.json().get("upload_id")
if upload_id:
# Abort the upload (without uploading any parts)
abort_response = integration_client.delete(
f"/api/v1/project/{project}/{package}/upload/{upload_id}",
headers={"Authorization": f"Bearer {api_key}"},
)
assert abort_response.status_code in [200, 204]
# Verify progress shows not_found after abort
progress_response = integration_client.get(
f"/api/v1/project/{project}/{package}/upload/{upload_id}/progress",
headers={"Authorization": f"Bearer {api_key}"},
)
assert progress_response.status_code == 200
assert progress_response.json()["status"] == "not_found"
@pytest.mark.integration
def test_abort_nonexistent_upload(self, integration_client, test_package):
"""Test aborting nonexistent upload returns appropriate error."""
project, package = test_package
# Get API key for auth
api_key_response = integration_client.post(
"/api/v1/auth/keys",
json={"name": "abort-nonexistent-key"},
)
assert api_key_response.status_code == 200
api_key = api_key_response.json()["key"]
response = integration_client.delete(
f"/api/v1/project/{project}/{package}/upload/nonexistent-upload-id",
headers={"Authorization": f"Bearer {api_key}"},
)
# Should return 404 or 200 (idempotent delete)
assert response.status_code in [200, 204, 404]
class TestUploadTimeout:
"""Tests for upload timeout handling."""
@pytest.mark.integration
def test_upload_with_short_timeout_succeeds_for_small_file(
self, integration_client, test_package
):
"""Test small file upload succeeds with reasonable timeout."""
project, package = test_package
content = b"small timeout test"
# httpx client should handle this quickly
result = upload_test_file(
integration_client, project, package, content, tag="timeout-small"
)
assert result["artifact_id"] is not None
@pytest.mark.integration
def test_upload_response_duration_under_timeout(
self, integration_client, test_package, sized_content
):
"""Test upload completes within reasonable time."""
project, package = test_package
content, _ = sized_content(SIZE_1MB, seed=800)
start = time.time()
result = upload_test_file(
integration_client, project, package, content, tag="timeout-check"
)
duration = time.time() - start
# 1MB should upload in well under 60 seconds on local
assert duration < 60
assert result["artifact_id"] is not None
class TestConcurrentUploads:
"""Tests for concurrent upload handling."""
@pytest.mark.integration
def test_concurrent_different_files(
self, integration_client, test_package, sized_content
):
"""Test concurrent uploads of different files succeed."""
from concurrent.futures import ThreadPoolExecutor, as_completed
project, package = test_package
# Get API key for auth
api_key_response = integration_client.post(
"/api/v1/auth/keys",
json={"name": "concurrent-diff-key"},
)
assert api_key_response.status_code == 200
api_key = api_key_response.json()["key"]
num_uploads = 3
results = []
errors = []
def upload_unique_file(idx):
try:
from httpx import Client
content, expected_hash = sized_content(SIZE_100KB, seed=900 + idx)
base_url = os.environ.get("ORCHARD_TEST_URL", "http://localhost:8080")
with Client(base_url=base_url, timeout=30.0) as client:
files = {
"file": (
f"concurrent-{idx}.bin",
io.BytesIO(content),
"application/octet-stream",
)
}
response = client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
data={"tag": f"concurrent-diff-{idx}"},
headers={"Authorization": f"Bearer {api_key}"},
)
if response.status_code == 200:
results.append((idx, response.json(), expected_hash))
else:
errors.append(f"Upload {idx}: {response.status_code} - {response.text}")
except Exception as e:
errors.append(f"Upload {idx}: {str(e)}")
with ThreadPoolExecutor(max_workers=num_uploads) as executor:
futures = [executor.submit(upload_unique_file, i) for i in range(num_uploads)]
for future in as_completed(futures):
pass
assert len(errors) == 0, f"Concurrent upload errors: {errors}"
assert len(results) == num_uploads
# Each upload should have unique artifact ID
artifact_ids = set(r[1]["artifact_id"] for r in results)
assert len(artifact_ids) == num_uploads
# Each should match expected hash
for idx, result, expected_hash in results:
assert result["artifact_id"] == expected_hash

View File

@@ -0,0 +1,583 @@
"""
Integration tests for upload/download with various file sizes.
Tests cover:
- Small files (0B - 100KB)
- Medium files (1MB - 50MB)
- Large files (100MB - 1GB) - marked as slow/large
- Exact chunk boundaries
- Data integrity verification across all sizes
"""
import pytest
import io
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
from tests.factories import (
compute_sha256,
upload_test_file,
generate_content,
generate_content_with_hash,
)
from tests.conftest import (
SIZE_1B,
SIZE_1KB,
SIZE_10KB,
SIZE_100KB,
SIZE_1MB,
SIZE_5MB,
SIZE_10MB,
SIZE_50MB,
SIZE_100MB,
SIZE_250MB,
SIZE_500MB,
SIZE_1GB,
CHUNK_SIZE,
MULTIPART_THRESHOLD,
)
class TestSmallFileSizes:
"""Tests for small file uploads/downloads (0B - 100KB)."""
@pytest.mark.integration
def test_upload_download_1_byte(self, integration_client, test_package, sized_content):
"""Test upload/download of 1 byte file."""
project, package = test_package
content, expected_hash = sized_content(SIZE_1B, seed=1)
result = upload_test_file(
integration_client, project, package, content,
filename="1byte.bin", tag="1byte"
)
assert result["artifact_id"] == expected_hash
assert result["size"] == SIZE_1B
# Download and verify
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/1byte",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert response.content == content
assert len(response.content) == SIZE_1B
@pytest.mark.integration
def test_upload_download_1kb(self, integration_client, test_package, sized_content):
"""Test upload/download of 1KB file."""
project, package = test_package
content, expected_hash = sized_content(SIZE_1KB, seed=2)
result = upload_test_file(
integration_client, project, package, content,
filename="1kb.bin", tag="1kb"
)
assert result["artifact_id"] == expected_hash
assert result["size"] == SIZE_1KB
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/1kb",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert response.content == content
@pytest.mark.integration
def test_upload_download_10kb(self, integration_client, test_package, sized_content):
"""Test upload/download of 10KB file."""
project, package = test_package
content, expected_hash = sized_content(SIZE_10KB, seed=3)
result = upload_test_file(
integration_client, project, package, content,
filename="10kb.bin", tag="10kb"
)
assert result["artifact_id"] == expected_hash
assert result["size"] == SIZE_10KB
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/10kb",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert response.content == content
@pytest.mark.integration
def test_upload_download_100kb(self, integration_client, test_package, sized_content):
"""Test upload/download of 100KB file."""
project, package = test_package
content, expected_hash = sized_content(SIZE_100KB, seed=4)
result = upload_test_file(
integration_client, project, package, content,
filename="100kb.bin", tag="100kb"
)
assert result["artifact_id"] == expected_hash
assert result["size"] == SIZE_100KB
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/100kb",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert response.content == content
class TestMediumFileSizes:
"""Tests for medium file uploads/downloads (1MB - 50MB)."""
@pytest.mark.integration
def test_upload_download_1mb(self, integration_client, test_package, sized_content):
"""Test upload/download of 1MB file."""
project, package = test_package
content, expected_hash = sized_content(SIZE_1MB, seed=10)
result = upload_test_file(
integration_client, project, package, content,
filename="1mb.bin", tag="1mb"
)
assert result["artifact_id"] == expected_hash
assert result["size"] == SIZE_1MB
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/1mb",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert len(response.content) == SIZE_1MB
assert compute_sha256(response.content) == expected_hash
@pytest.mark.integration
def test_upload_download_5mb(self, integration_client, test_package, sized_content):
"""Test upload/download of 5MB file (multipart threshold boundary area)."""
project, package = test_package
content, expected_hash = sized_content(SIZE_5MB, seed=11)
result = upload_test_file(
integration_client, project, package, content,
filename="5mb.bin", tag="5mb"
)
assert result["artifact_id"] == expected_hash
assert result["size"] == SIZE_5MB
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/5mb",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert len(response.content) == SIZE_5MB
assert compute_sha256(response.content) == expected_hash
@pytest.mark.integration
@pytest.mark.slow
def test_upload_download_10mb(self, integration_client, test_package, sized_content):
"""Test upload/download of 10MB file."""
project, package = test_package
content, expected_hash = sized_content(SIZE_10MB, seed=12)
result = upload_test_file(
integration_client, project, package, content,
filename="10mb.bin", tag="10mb"
)
assert result["artifact_id"] == expected_hash
assert result["size"] == SIZE_10MB
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/10mb",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert len(response.content) == SIZE_10MB
assert compute_sha256(response.content) == expected_hash
@pytest.mark.integration
@pytest.mark.slow
def test_upload_download_50mb(self, integration_client, test_package, sized_content):
"""Test upload/download of 50MB file."""
project, package = test_package
content, expected_hash = sized_content(SIZE_50MB, seed=13)
start_time = time.time()
result = upload_test_file(
integration_client, project, package, content,
filename="50mb.bin", tag="50mb"
)
upload_time = time.time() - start_time
assert result["artifact_id"] == expected_hash
assert result["size"] == SIZE_50MB
start_time = time.time()
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/50mb",
params={"mode": "proxy"},
)
download_time = time.time() - start_time
assert response.status_code == 200
assert len(response.content) == SIZE_50MB
assert compute_sha256(response.content) == expected_hash
# Log timing for performance tracking
print(f"\n50MB upload: {upload_time:.2f}s, download: {download_time:.2f}s")
class TestLargeFileSizes:
"""Tests for large file uploads/downloads (100MB - 1GB).
These tests are marked as slow and large, skipped by default.
Run with: pytest -m "large" to include these tests.
"""
@pytest.mark.integration
@pytest.mark.slow
@pytest.mark.large
def test_upload_download_100mb(self, integration_client, test_package, sized_content):
"""Test upload/download of 100MB file (multipart threshold)."""
project, package = test_package
content, expected_hash = sized_content(SIZE_100MB, seed=100)
start_time = time.time()
result = upload_test_file(
integration_client, project, package, content,
filename="100mb.bin", tag="100mb"
)
upload_time = time.time() - start_time
assert result["artifact_id"] == expected_hash
assert result["size"] == SIZE_100MB
start_time = time.time()
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/100mb",
params={"mode": "proxy"},
)
download_time = time.time() - start_time
assert response.status_code == 200
assert len(response.content) == SIZE_100MB
assert compute_sha256(response.content) == expected_hash
print(f"\n100MB upload: {upload_time:.2f}s, download: {download_time:.2f}s")
@pytest.mark.integration
@pytest.mark.slow
@pytest.mark.large
def test_upload_download_250mb(self, integration_client, test_package, sized_content):
"""Test upload/download of 250MB file."""
project, package = test_package
content, expected_hash = sized_content(SIZE_250MB, seed=250)
start_time = time.time()
result = upload_test_file(
integration_client, project, package, content,
filename="250mb.bin", tag="250mb"
)
upload_time = time.time() - start_time
assert result["artifact_id"] == expected_hash
assert result["size"] == SIZE_250MB
start_time = time.time()
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/250mb",
params={"mode": "proxy"},
)
download_time = time.time() - start_time
assert response.status_code == 200
assert len(response.content) == SIZE_250MB
assert compute_sha256(response.content) == expected_hash
print(f"\n250MB upload: {upload_time:.2f}s, download: {download_time:.2f}s")
@pytest.mark.integration
@pytest.mark.slow
@pytest.mark.large
def test_upload_download_500mb(self, integration_client, test_package, sized_content):
"""Test upload/download of 500MB file."""
project, package = test_package
content, expected_hash = sized_content(SIZE_500MB, seed=500)
start_time = time.time()
result = upload_test_file(
integration_client, project, package, content,
filename="500mb.bin", tag="500mb"
)
upload_time = time.time() - start_time
assert result["artifact_id"] == expected_hash
assert result["size"] == SIZE_500MB
start_time = time.time()
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/500mb",
params={"mode": "proxy"},
)
download_time = time.time() - start_time
assert response.status_code == 200
assert len(response.content) == SIZE_500MB
assert compute_sha256(response.content) == expected_hash
print(f"\n500MB upload: {upload_time:.2f}s, download: {download_time:.2f}s")
@pytest.mark.integration
@pytest.mark.slow
@pytest.mark.large
def test_upload_download_1gb(self, integration_client, test_package, sized_content):
"""Test upload/download of 1GB file.
This test may take several minutes depending on network/disk speed.
"""
project, package = test_package
content, expected_hash = sized_content(SIZE_1GB, seed=1024)
start_time = time.time()
result = upload_test_file(
integration_client, project, package, content,
filename="1gb.bin", tag="1gb"
)
upload_time = time.time() - start_time
assert result["artifact_id"] == expected_hash
assert result["size"] == SIZE_1GB
start_time = time.time()
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/1gb",
params={"mode": "proxy"},
)
download_time = time.time() - start_time
assert response.status_code == 200
assert len(response.content) == SIZE_1GB
assert compute_sha256(response.content) == expected_hash
print(f"\n1GB upload: {upload_time:.2f}s, download: {download_time:.2f}s")
class TestChunkBoundaries:
"""Tests for exact chunk size boundaries."""
@pytest.mark.integration
def test_upload_download_at_chunk_size(self, integration_client, test_package, sized_content):
"""Test upload/download at exact chunk size (64KB)."""
project, package = test_package
content, expected_hash = sized_content(CHUNK_SIZE, seed=64)
result = upload_test_file(
integration_client, project, package, content,
filename="chunk.bin", tag="chunk-exact"
)
assert result["artifact_id"] == expected_hash
assert result["size"] == CHUNK_SIZE
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/chunk-exact",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert response.content == content
@pytest.mark.integration
def test_upload_download_chunk_size_plus_1(self, integration_client, test_package, sized_content):
"""Test upload/download at chunk size + 1 byte."""
project, package = test_package
size = CHUNK_SIZE + 1
content, expected_hash = sized_content(size, seed=65)
result = upload_test_file(
integration_client, project, package, content,
filename="chunk_plus.bin", tag="chunk-plus"
)
assert result["artifact_id"] == expected_hash
assert result["size"] == size
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/chunk-plus",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert response.content == content
@pytest.mark.integration
def test_upload_download_chunk_size_minus_1(self, integration_client, test_package, sized_content):
"""Test upload/download at chunk size - 1 byte."""
project, package = test_package
size = CHUNK_SIZE - 1
content, expected_hash = sized_content(size, seed=63)
result = upload_test_file(
integration_client, project, package, content,
filename="chunk_minus.bin", tag="chunk-minus"
)
assert result["artifact_id"] == expected_hash
assert result["size"] == size
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/chunk-minus",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert response.content == content
@pytest.mark.integration
def test_upload_download_multiple_chunks(self, integration_client, test_package, sized_content):
"""Test upload/download spanning multiple chunks."""
project, package = test_package
size = CHUNK_SIZE * 3 + 1000 # 3 full chunks + partial
content, expected_hash = sized_content(size, seed=300)
result = upload_test_file(
integration_client, project, package, content,
filename="multi_chunk.bin", tag="multi-chunk"
)
assert result["artifact_id"] == expected_hash
assert result["size"] == size
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/multi-chunk",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert response.content == content
class TestDataIntegrity:
"""Tests for data integrity with various content types."""
@pytest.mark.integration
def test_binary_content_integrity(self, integration_client, test_package):
"""Test binary content (all byte values 0-255) integrity."""
project, package = test_package
# Content with all 256 possible byte values
content = bytes(range(256)) * 100 # 25.6KB
expected_hash = compute_sha256(content)
result = upload_test_file(
integration_client, project, package, content,
filename="binary.bin", tag="binary"
)
assert result["artifact_id"] == expected_hash
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/binary",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert response.content == content
@pytest.mark.integration
def test_text_content_integrity(self, integration_client, test_package):
"""Test UTF-8 text content integrity."""
project, package = test_package
content = "Hello, World! 你好世界 🌍 مرحبا العالم".encode("utf-8")
expected_hash = compute_sha256(content)
result = upload_test_file(
integration_client, project, package, content,
filename="text.txt", tag="text"
)
assert result["artifact_id"] == expected_hash
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/text",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert response.content == content
assert response.content.decode("utf-8") == "Hello, World! 你好世界 🌍 مرحبا العالم"
@pytest.mark.integration
def test_null_bytes_content_integrity(self, integration_client, test_package):
"""Test content with null bytes."""
project, package = test_package
content = b"before\x00null\x00bytes\x00after"
expected_hash = compute_sha256(content)
result = upload_test_file(
integration_client, project, package, content,
filename="nulls.bin", tag="nulls"
)
assert result["artifact_id"] == expected_hash
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/nulls",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert response.content == content
assert b"\x00" in response.content
@pytest.mark.integration
def test_unicode_filename_integrity(self, integration_client, test_package):
"""Test file with unicode filename."""
project, package = test_package
content = b"unicode filename test"
expected_hash = compute_sha256(content)
result = upload_test_file(
integration_client, project, package, content,
filename="文件名.txt", tag="unicode-name"
)
assert result["artifact_id"] == expected_hash
assert result["original_name"] == "文件名.txt"
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/unicode-name",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert response.content == content
@pytest.mark.integration
def test_compressed_content_integrity(self, integration_client, test_package):
"""Test gzip-compressed content integrity."""
import gzip
project, package = test_package
original = b"This is some text that will be compressed " * 100
content = gzip.compress(original)
expected_hash = compute_sha256(content)
result = upload_test_file(
integration_client, project, package, content,
filename="data.gz", tag="compressed"
)
assert result["artifact_id"] == expected_hash
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/compressed",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert response.content == content
# Verify we can decompress
assert gzip.decompress(response.content) == original
@pytest.mark.integration
def test_hash_verification_matches(self, integration_client, test_package, sized_content):
"""Test that computed hash matches artifact_id for various sizes."""
project, package = test_package
sizes = [SIZE_1B, SIZE_1KB, SIZE_10KB, SIZE_100KB, SIZE_1MB]
for i, size in enumerate(sizes):
content, expected_hash = sized_content(size, seed=1000 + i)
result = upload_test_file(
integration_client, project, package, content,
filename=f"hash_test_{size}.bin", tag=f"hash-{size}"
)
# Verify artifact_id matches expected hash
assert result["artifact_id"] == expected_hash
# Download and verify hash of downloaded content
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/hash-{size}",
params={"mode": "proxy"},
)
downloaded_hash = compute_sha256(response.content)
assert downloaded_hash == expected_hash

View File

@@ -0,0 +1,535 @@
"""
Integration tests for streaming download functionality.
Tests cover:
- HTTP Range requests (partial downloads, resume)
- Conditional requests (If-None-Match, If-Modified-Since)
- Caching headers (Cache-Control, Last-Modified, Accept-Ranges)
- Large file streaming
- Download modes (proxy, redirect, presigned)
"""
import pytest
import io
import time
from email.utils import formatdate
from tests.factories import (
compute_sha256,
upload_test_file,
)
from tests.conftest import (
SIZE_1KB,
SIZE_100KB,
SIZE_1MB,
)
class TestRangeRequests:
"""Tests for HTTP Range request support (partial downloads)."""
@pytest.mark.integration
def test_range_request_first_bytes(self, integration_client, test_package):
"""Test range request for first N bytes."""
project, package = test_package
content = b"0123456789" * 100 # 1000 bytes
upload_test_file(integration_client, project, package, content, tag="range-test")
# Request first 10 bytes
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/range-test",
params={"mode": "proxy"},
headers={"Range": "bytes=0-9"},
)
assert response.status_code == 206 # Partial Content
assert response.content == b"0123456789"
assert "Content-Range" in response.headers
assert response.headers["Content-Range"].startswith("bytes 0-9/")
@pytest.mark.integration
def test_range_request_middle_bytes(self, integration_client, test_package):
"""Test range request for bytes in the middle."""
project, package = test_package
content = b"ABCDEFGHIJKLMNOPQRSTUVWXYZ"
upload_test_file(integration_client, project, package, content, tag="range-mid")
# Request bytes 10-19 (KLMNOPQRST)
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/range-mid",
params={"mode": "proxy"},
headers={"Range": "bytes=10-19"},
)
assert response.status_code == 206
assert response.content == b"KLMNOPQRST"
@pytest.mark.integration
def test_range_request_suffix_bytes(self, integration_client, test_package):
"""Test range request for last N bytes (suffix range)."""
project, package = test_package
content = b"0123456789ABCDEF" # 16 bytes
upload_test_file(integration_client, project, package, content, tag="range-suffix")
# Request last 4 bytes
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/range-suffix",
params={"mode": "proxy"},
headers={"Range": "bytes=-4"},
)
assert response.status_code == 206
assert response.content == b"CDEF"
@pytest.mark.integration
def test_range_request_open_ended(self, integration_client, test_package):
"""Test range request from offset to end."""
project, package = test_package
content = b"0123456789"
upload_test_file(integration_client, project, package, content, tag="range-open")
# Request from byte 5 to end
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/range-open",
params={"mode": "proxy"},
headers={"Range": "bytes=5-"},
)
assert response.status_code == 206
assert response.content == b"56789"
@pytest.mark.integration
def test_range_request_includes_accept_ranges_header(
self, integration_client, test_package
):
"""Test that range requests include Accept-Ranges header."""
project, package = test_package
content = b"test content"
upload_test_file(integration_client, project, package, content, tag="accept-ranges")
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/accept-ranges",
params={"mode": "proxy"},
headers={"Range": "bytes=0-4"},
)
assert response.status_code == 206
assert response.headers.get("Accept-Ranges") == "bytes"
@pytest.mark.integration
def test_full_download_advertises_accept_ranges(
self, integration_client, test_package
):
"""Test that full downloads advertise range support."""
project, package = test_package
content = b"test content"
upload_test_file(integration_client, project, package, content, tag="full-accept")
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/full-accept",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert response.headers.get("Accept-Ranges") == "bytes"
class TestConditionalRequests:
"""Tests for conditional request handling (304 Not Modified)."""
@pytest.mark.integration
def test_if_none_match_returns_304(self, integration_client, test_package):
"""Test If-None-Match with matching ETag returns 304."""
project, package = test_package
content = b"conditional request test content"
expected_hash = compute_sha256(content)
upload_test_file(integration_client, project, package, content, tag="cond-etag")
# Request with matching ETag
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/cond-etag",
params={"mode": "proxy"},
headers={"If-None-Match": f'"{expected_hash}"'},
)
assert response.status_code == 304
assert response.content == b"" # No body for 304
@pytest.mark.integration
def test_if_none_match_without_quotes(self, integration_client, test_package):
"""Test If-None-Match works with or without quotes."""
project, package = test_package
content = b"etag no quotes test"
expected_hash = compute_sha256(content)
upload_test_file(integration_client, project, package, content, tag="cond-noquote")
# Request with ETag without quotes
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/cond-noquote",
params={"mode": "proxy"},
headers={"If-None-Match": expected_hash},
)
assert response.status_code == 304
@pytest.mark.integration
def test_if_none_match_mismatch_returns_200(self, integration_client, test_package):
"""Test If-None-Match with non-matching ETag returns 200."""
project, package = test_package
content = b"etag mismatch test"
upload_test_file(integration_client, project, package, content, tag="cond-mismatch")
# Request with different ETag
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/cond-mismatch",
params={"mode": "proxy"},
headers={"If-None-Match": '"different-etag-value"'},
)
assert response.status_code == 200
assert response.content == content
@pytest.mark.integration
def test_if_modified_since_returns_304(self, integration_client, test_package):
"""Test If-Modified-Since with future date returns 304."""
project, package = test_package
content = b"modified since test"
upload_test_file(integration_client, project, package, content, tag="cond-modified")
# Request with future date (artifact was definitely created before this)
future_date = formatdate(time.time() + 86400, usegmt=True) # Tomorrow
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/cond-modified",
params={"mode": "proxy"},
headers={"If-Modified-Since": future_date},
)
assert response.status_code == 304
@pytest.mark.integration
def test_if_modified_since_old_date_returns_200(
self, integration_client, test_package
):
"""Test If-Modified-Since with old date returns 200."""
project, package = test_package
content = b"old date test"
upload_test_file(integration_client, project, package, content, tag="cond-old")
# Request with old date (2020-01-01)
old_date = "Wed, 01 Jan 2020 00:00:00 GMT"
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/cond-old",
params={"mode": "proxy"},
headers={"If-Modified-Since": old_date},
)
assert response.status_code == 200
assert response.content == content
@pytest.mark.integration
def test_304_includes_etag(self, integration_client, test_package):
"""Test 304 response includes ETag header."""
project, package = test_package
content = b"304 etag test"
expected_hash = compute_sha256(content)
upload_test_file(integration_client, project, package, content, tag="304-etag")
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/304-etag",
params={"mode": "proxy"},
headers={"If-None-Match": f'"{expected_hash}"'},
)
assert response.status_code == 304
assert response.headers.get("ETag") == f'"{expected_hash}"'
@pytest.mark.integration
def test_304_includes_cache_control(self, integration_client, test_package):
"""Test 304 response includes Cache-Control header."""
project, package = test_package
content = b"304 cache test"
expected_hash = compute_sha256(content)
upload_test_file(integration_client, project, package, content, tag="304-cache")
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/304-cache",
params={"mode": "proxy"},
headers={"If-None-Match": f'"{expected_hash}"'},
)
assert response.status_code == 304
assert "immutable" in response.headers.get("Cache-Control", "")
class TestCachingHeaders:
"""Tests for caching headers on download responses."""
@pytest.mark.integration
def test_download_includes_cache_control(self, integration_client, test_package):
"""Test download response includes Cache-Control header."""
project, package = test_package
content = b"cache control test"
upload_test_file(integration_client, project, package, content, tag="cache-ctl")
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/cache-ctl",
params={"mode": "proxy"},
)
assert response.status_code == 200
cache_control = response.headers.get("Cache-Control", "")
assert "public" in cache_control
assert "immutable" in cache_control
assert "max-age" in cache_control
@pytest.mark.integration
def test_download_includes_last_modified(self, integration_client, test_package):
"""Test download response includes Last-Modified header."""
project, package = test_package
content = b"last modified test"
upload_test_file(integration_client, project, package, content, tag="last-mod")
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/last-mod",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert "Last-Modified" in response.headers
# Should be in RFC 7231 format
last_modified = response.headers["Last-Modified"]
assert "GMT" in last_modified
@pytest.mark.integration
def test_download_includes_etag(self, integration_client, test_package):
"""Test download response includes ETag header."""
project, package = test_package
content = b"etag header test"
expected_hash = compute_sha256(content)
upload_test_file(integration_client, project, package, content, tag="etag-hdr")
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/etag-hdr",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert response.headers.get("ETag") == f'"{expected_hash}"'
class TestDownloadResume:
"""Tests for download resume functionality using range requests."""
@pytest.mark.integration
def test_resume_download_after_partial(self, integration_client, test_package):
"""Test resuming download from where it left off."""
project, package = test_package
content = b"ABCDEFGHIJ" * 100 # 1000 bytes
upload_test_file(integration_client, project, package, content, tag="resume-test")
# Simulate partial download (first 500 bytes)
response1 = integration_client.get(
f"/api/v1/project/{project}/{package}/+/resume-test",
params={"mode": "proxy"},
headers={"Range": "bytes=0-499"},
)
assert response1.status_code == 206
first_half = response1.content
assert len(first_half) == 500
# Resume from byte 500
response2 = integration_client.get(
f"/api/v1/project/{project}/{package}/+/resume-test",
params={"mode": "proxy"},
headers={"Range": "bytes=500-"},
)
assert response2.status_code == 206
second_half = response2.content
assert len(second_half) == 500
# Combine and verify
combined = first_half + second_half
assert combined == content
@pytest.mark.integration
def test_resume_with_etag_verification(self, integration_client, test_package):
"""Test that resumed download can verify content hasn't changed."""
project, package = test_package
content = b"resume etag verification test content"
expected_hash = compute_sha256(content)
upload_test_file(integration_client, project, package, content, tag="resume-etag")
# Get ETag from first request
response1 = integration_client.get(
f"/api/v1/project/{project}/{package}/+/resume-etag",
params={"mode": "proxy"},
headers={"Range": "bytes=0-9"},
)
assert response1.status_code == 206
etag = response1.headers.get("ETag")
assert etag == f'"{expected_hash}"'
# Resume with If-Match to ensure content hasn't changed
# (Note: If-Match would fail and return 412 if content changed)
response2 = integration_client.get(
f"/api/v1/project/{project}/{package}/+/resume-etag",
params={"mode": "proxy"},
headers={"Range": "bytes=10-"},
)
assert response2.status_code == 206
# ETag should be the same
assert response2.headers.get("ETag") == etag
class TestLargeFileStreaming:
"""Tests for streaming large files."""
@pytest.mark.integration
def test_stream_1mb_file(self, integration_client, test_package, sized_content):
"""Test streaming a 1MB file."""
project, package = test_package
content, expected_hash = sized_content(SIZE_1MB, seed=500)
upload_test_file(integration_client, project, package, content, tag="stream-1mb")
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/stream-1mb",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert len(response.content) == SIZE_1MB
assert compute_sha256(response.content) == expected_hash
@pytest.mark.integration
def test_stream_large_file_has_correct_headers(
self, integration_client, test_package, sized_content
):
"""Test that large file streaming has correct headers."""
project, package = test_package
content, expected_hash = sized_content(SIZE_100KB, seed=501)
upload_test_file(integration_client, project, package, content, tag="stream-hdr")
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/stream-hdr",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert int(response.headers.get("Content-Length", 0)) == SIZE_100KB
assert response.headers.get("X-Checksum-SHA256") == expected_hash
assert response.headers.get("Accept-Ranges") == "bytes"
@pytest.mark.integration
def test_range_request_on_large_file(
self, integration_client, test_package, sized_content
):
"""Test range request on a larger file."""
project, package = test_package
content, _ = sized_content(SIZE_100KB, seed=502)
upload_test_file(integration_client, project, package, content, tag="range-large")
# Request a slice from the middle
start = 50000
end = 50999
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/range-large",
params={"mode": "proxy"},
headers={"Range": f"bytes={start}-{end}"},
)
assert response.status_code == 206
assert len(response.content) == 1000
assert response.content == content[start : end + 1]
class TestDownloadModes:
"""Tests for different download modes."""
@pytest.mark.integration
def test_proxy_mode_streams_content(self, integration_client, test_package):
"""Test proxy mode streams content through backend."""
project, package = test_package
content = b"proxy mode test content"
upload_test_file(integration_client, project, package, content, tag="mode-proxy")
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/mode-proxy",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert response.content == content
@pytest.mark.integration
def test_presigned_mode_returns_url(self, integration_client, test_package):
"""Test presigned mode returns JSON with URL."""
project, package = test_package
content = b"presigned mode test"
upload_test_file(integration_client, project, package, content, tag="mode-presign")
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/mode-presign",
params={"mode": "presigned"},
)
assert response.status_code == 200
data = response.json()
assert "url" in data
assert "expires_at" in data
assert data["url"].startswith("http")
@pytest.mark.integration
def test_redirect_mode_returns_302(self, integration_client, test_package):
"""Test redirect mode returns 302 to presigned URL."""
project, package = test_package
content = b"redirect mode test"
upload_test_file(integration_client, project, package, content, tag="mode-redir")
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/mode-redir",
params={"mode": "redirect"},
follow_redirects=False,
)
assert response.status_code == 302
assert "Location" in response.headers
class TestIntegrityDuringStreaming:
"""Tests for data integrity during streaming downloads."""
@pytest.mark.integration
def test_checksum_header_matches_content(self, integration_client, test_package):
"""Test X-Checksum-SHA256 header matches actual downloaded content."""
project, package = test_package
content = b"integrity check content"
expected_hash = compute_sha256(content)
upload_test_file(integration_client, project, package, content, tag="integrity")
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/integrity",
params={"mode": "proxy"},
)
assert response.status_code == 200
header_hash = response.headers.get("X-Checksum-SHA256")
actual_hash = compute_sha256(response.content)
assert header_hash == expected_hash
assert actual_hash == expected_hash
assert header_hash == actual_hash
@pytest.mark.integration
def test_etag_matches_content_hash(self, integration_client, test_package):
"""Test ETag header matches content hash."""
project, package = test_package
content = b"etag integrity test"
expected_hash = compute_sha256(content)
upload_test_file(integration_client, project, package, content, tag="etag-int")
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/etag-int",
params={"mode": "proxy"},
)
assert response.status_code == 200
etag = response.headers.get("ETag", "").strip('"')
actual_hash = compute_sha256(response.content)
assert etag == expected_hash
assert actual_hash == expected_hash
@pytest.mark.integration
def test_digest_header_present(self, integration_client, test_package):
"""Test Digest header is present in RFC 3230 format."""
project, package = test_package
content = b"digest header test"
upload_test_file(integration_client, project, package, content, tag="digest")
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/digest",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert "Digest" in response.headers
assert response.headers["Digest"].startswith("sha-256=")

View File

@@ -10,6 +10,7 @@ Tests cover:
- S3 storage verification
"""
import os
import pytest
import io
import threading
@@ -25,6 +26,19 @@ from tests.factories import (
class TestUploadBasics:
"""Tests for basic upload functionality."""
@pytest.mark.integration
def test_upload_returns_200(self, integration_client, test_package):
"""Test upload with valid file returns 200."""
project, package = test_package
content = b"valid file upload test"
files = {"file": ("test.bin", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
)
assert response.status_code == 200
@pytest.mark.integration
def test_upload_returns_artifact_id(self, integration_client, test_package):
"""Test upload returns the artifact ID (SHA256 hash)."""
@@ -101,6 +115,83 @@ class TestUploadBasics:
assert "created_at" in result
assert result["created_at"] is not None
@pytest.mark.integration
def test_upload_without_tag_succeeds(self, integration_client, test_package):
"""Test upload without tag succeeds (no tag created)."""
project, package = test_package
content = b"upload without tag test"
expected_hash = compute_sha256(content)
files = {"file": ("no_tag.bin", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
# No tag parameter
)
assert response.status_code == 200
result = response.json()
assert result["artifact_id"] == expected_hash
# Verify no tag was created - list tags and check
tags_response = integration_client.get(
f"/api/v1/project/{project}/{package}/tags"
)
assert tags_response.status_code == 200
tags = tags_response.json()
# Filter for tags pointing to this artifact
artifact_tags = [t for t in tags.get("items", tags) if t.get("artifact_id") == expected_hash]
assert len(artifact_tags) == 0, "Tag should not be created when not specified"
@pytest.mark.integration
def test_upload_creates_artifact_in_database(self, integration_client, test_package):
"""Test upload creates artifact record in database."""
project, package = test_package
content = b"database artifact test"
expected_hash = compute_sha256(content)
upload_test_file(integration_client, project, package, content)
# Verify artifact exists via API
response = integration_client.get(f"/api/v1/artifact/{expected_hash}")
assert response.status_code == 200
artifact = response.json()
assert artifact["id"] == expected_hash
assert artifact["size"] == len(content)
@pytest.mark.integration
@pytest.mark.requires_direct_s3
def test_upload_creates_object_in_s3(self, integration_client, test_package):
"""Test upload creates object in S3 storage."""
project, package = test_package
content = b"s3 object creation test"
expected_hash = compute_sha256(content)
upload_test_file(integration_client, project, package, content)
# Verify S3 object exists
assert s3_object_exists(expected_hash), "S3 object should exist after upload"
@pytest.mark.integration
def test_upload_with_tag_creates_tag_record(self, integration_client, test_package):
"""Test upload with tag creates tag record."""
project, package = test_package
content = b"tag creation test"
expected_hash = compute_sha256(content)
tag_name = "my-tag-v1"
upload_test_file(
integration_client, project, package, content, tag=tag_name
)
# Verify tag exists
tags_response = integration_client.get(
f"/api/v1/project/{project}/{package}/tags"
)
assert tags_response.status_code == 200
tags = tags_response.json()
tag_names = [t["name"] for t in tags.get("items", tags)]
assert tag_name in tag_names
class TestDuplicateUploads:
"""Tests for duplicate upload deduplication behavior."""
@@ -248,6 +339,23 @@ class TestDownload:
assert response.status_code == 200
assert response.content == original_content
@pytest.mark.integration
def test_download_by_tag_prefix(self, integration_client, test_package):
"""Test downloading artifact using tag: prefix."""
project, package = test_package
original_content = b"download by tag prefix test"
upload_test_file(
integration_client, project, package, original_content, tag="prefix-tag"
)
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/tag:prefix-tag",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert response.content == original_content
@pytest.mark.integration
def test_download_nonexistent_tag(self, integration_client, test_package):
"""Test downloading nonexistent tag returns 404."""
@@ -258,6 +366,33 @@ class TestDownload:
)
assert response.status_code == 404
@pytest.mark.integration
def test_download_nonexistent_artifact(self, integration_client, test_package):
"""Test downloading nonexistent artifact ID returns 404."""
project, package = test_package
fake_hash = "0" * 64
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/artifact:{fake_hash}"
)
assert response.status_code == 404
@pytest.mark.integration
def test_download_from_nonexistent_project(self, integration_client, unique_test_id):
"""Test downloading from nonexistent project returns 404."""
response = integration_client.get(
f"/api/v1/project/nonexistent-project-{unique_test_id}/somepackage/+/sometag"
)
assert response.status_code == 404
@pytest.mark.integration
def test_download_from_nonexistent_package(self, integration_client, test_project, unique_test_id):
"""Test downloading from nonexistent package returns 404."""
response = integration_client.get(
f"/api/v1/project/{test_project}/nonexistent-package-{unique_test_id}/+/sometag"
)
assert response.status_code == 404
@pytest.mark.integration
def test_content_matches_original(self, integration_client, test_package):
"""Test downloaded content matches original exactly."""
@@ -275,6 +410,111 @@ class TestDownload:
assert response.content == original_content
class TestDownloadHeaders:
"""Tests for download response headers."""
@pytest.mark.integration
def test_download_content_type_header(self, integration_client, test_package):
"""Test download returns correct Content-Type header."""
project, package = test_package
content = b"content type header test"
upload_test_file(
integration_client, project, package, content,
filename="test.txt", tag="content-type-test"
)
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/content-type-test",
params={"mode": "proxy"},
)
assert response.status_code == 200
# Content-Type should be set (either text/plain or application/octet-stream)
assert "content-type" in response.headers
@pytest.mark.integration
def test_download_content_length_header(self, integration_client, test_package):
"""Test download returns correct Content-Length header."""
project, package = test_package
content = b"content length header test - exactly 41 bytes!"
expected_length = len(content)
upload_test_file(
integration_client, project, package, content, tag="content-length-test"
)
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/content-length-test",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert "content-length" in response.headers
assert int(response.headers["content-length"]) == expected_length
@pytest.mark.integration
def test_download_content_disposition_header(self, integration_client, test_package):
"""Test download returns correct Content-Disposition header."""
project, package = test_package
content = b"content disposition test"
filename = "my-test-file.bin"
upload_test_file(
integration_client, project, package, content,
filename=filename, tag="disposition-test"
)
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/disposition-test",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert "content-disposition" in response.headers
disposition = response.headers["content-disposition"]
assert "attachment" in disposition
assert filename in disposition
@pytest.mark.integration
def test_download_checksum_headers(self, integration_client, test_package):
"""Test download returns checksum headers."""
project, package = test_package
content = b"checksum header test content"
expected_hash = compute_sha256(content)
upload_test_file(
integration_client, project, package, content, tag="checksum-headers"
)
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/checksum-headers",
params={"mode": "proxy"},
)
assert response.status_code == 200
# Check for checksum headers
assert "x-checksum-sha256" in response.headers
assert response.headers["x-checksum-sha256"] == expected_hash
@pytest.mark.integration
def test_download_etag_header(self, integration_client, test_package):
"""Test download returns ETag header (artifact ID)."""
project, package = test_package
content = b"etag header test"
expected_hash = compute_sha256(content)
upload_test_file(
integration_client, project, package, content, tag="etag-test"
)
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/etag-test",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert "etag" in response.headers
# ETag should contain the artifact ID (hash)
etag = response.headers["etag"].strip('"')
assert etag == expected_hash
class TestConcurrentUploads:
"""Tests for concurrent upload handling."""
@@ -301,7 +541,7 @@ class TestConcurrentUploads:
try:
from httpx import Client
base_url = "http://localhost:8080"
base_url = os.environ.get("ORCHARD_TEST_URL", "http://localhost:8080")
with Client(base_url=base_url, timeout=30.0) as client:
files = {
"file": (
@@ -397,6 +637,7 @@ class TestUploadFailureCleanup:
"""Tests for cleanup when uploads fail."""
@pytest.mark.integration
@pytest.mark.requires_direct_s3
def test_upload_failure_invalid_project_no_orphaned_s3(
self, integration_client, unique_test_id
):
@@ -419,6 +660,7 @@ class TestUploadFailureCleanup:
)
@pytest.mark.integration
@pytest.mark.requires_direct_s3
def test_upload_failure_invalid_package_no_orphaned_s3(
self, integration_client, test_project, unique_test_id
):
@@ -466,6 +708,7 @@ class TestS3StorageVerification:
"""Tests to verify S3 storage behavior."""
@pytest.mark.integration
@pytest.mark.requires_direct_s3
def test_s3_single_object_after_duplicates(
self, integration_client, test_package, unique_test_id
):
@@ -521,6 +764,7 @@ class TestSecurityPathTraversal:
"""
@pytest.mark.integration
@pytest.mark.requires_direct_s3
def test_path_traversal_in_filename_stored_safely(
self, integration_client, test_package
):

View File

@@ -0,0 +1,347 @@
"""
Integration tests for package version API endpoints.
Tests cover:
- Version creation via upload
- Version auto-detection from filename
- Version listing and retrieval
- Download by version prefix
- Version deletion
"""
import pytest
import io
from tests.factories import (
compute_sha256,
upload_test_file,
)
class TestVersionCreation:
"""Tests for creating versions via upload."""
@pytest.mark.integration
def test_upload_with_explicit_version(self, integration_client, test_package):
"""Test upload with explicit version parameter creates version record."""
project, package = test_package
content = b"version creation test"
expected_hash = compute_sha256(content)
files = {"file": ("app.tar.gz", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
data={"version": "1.0.0"},
)
assert response.status_code == 200
result = response.json()
assert result["artifact_id"] == expected_hash
assert result.get("version") == "1.0.0"
assert result.get("version_source") == "explicit"
@pytest.mark.integration
def test_upload_with_version_and_tag(self, integration_client, test_package):
"""Test upload with both version and tag creates both records."""
project, package = test_package
content = b"version and tag test"
files = {"file": ("app.tar.gz", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
data={"version": "2.0.0", "tag": "latest"},
)
assert response.status_code == 200
result = response.json()
assert result.get("version") == "2.0.0"
# Verify tag was also created
tags_response = integration_client.get(
f"/api/v1/project/{project}/{package}/tags"
)
assert tags_response.status_code == 200
tags = tags_response.json()
tag_names = [t["name"] for t in tags.get("items", tags)]
assert "latest" in tag_names
@pytest.mark.integration
def test_duplicate_version_same_content_succeeds(self, integration_client, test_package):
"""Test uploading same version with same content succeeds (deduplication)."""
project, package = test_package
content = b"version dedup test"
# First upload with version
files1 = {"file": ("app1.tar.gz", io.BytesIO(content), "application/octet-stream")}
response1 = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files1,
data={"version": "3.0.0"},
)
assert response1.status_code == 200
# Second upload with same version and same content succeeds
files2 = {"file": ("app2.tar.gz", io.BytesIO(content), "application/octet-stream")}
response2 = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files2,
data={"version": "3.0.0"},
)
# This succeeds because it's the same artifact (deduplication)
assert response2.status_code == 200
class TestVersionAutoDetection:
"""Tests for automatic version detection from filename."""
@pytest.mark.integration
def test_version_detected_from_filename_tarball(self, integration_client, test_package):
"""Test version is auto-detected from tarball filename or metadata."""
project, package = test_package
content = b"auto detect version tarball"
files = {"file": ("myapp-1.2.3.tar.gz", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
)
assert response.status_code == 200
result = response.json()
assert result.get("version") == "1.2.3"
# Version source can be 'filename' or 'metadata' depending on detection order
assert result.get("version_source") in ["filename", "metadata"]
@pytest.mark.integration
def test_version_detected_from_filename_zip(self, integration_client, test_package):
"""Test version is auto-detected from zip filename."""
project, package = test_package
content = b"auto detect version zip"
files = {"file": ("package-2.0.0.zip", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
)
assert response.status_code == 200
result = response.json()
assert result.get("version") == "2.0.0"
assert result.get("version_source") == "filename"
@pytest.mark.integration
def test_explicit_version_overrides_filename(self, integration_client, test_package):
"""Test explicit version parameter overrides filename detection."""
project, package = test_package
content = b"explicit override test"
files = {"file": ("myapp-1.0.0.tar.gz", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
data={"version": "9.9.9"},
)
assert response.status_code == 200
result = response.json()
assert result.get("version") == "9.9.9"
assert result.get("version_source") == "explicit"
@pytest.mark.integration
def test_no_version_detected_from_plain_filename(self, integration_client, test_package):
"""Test no version is created for filenames without version pattern."""
project, package = test_package
content = b"no version in filename"
files = {"file": ("plain-file.bin", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
)
assert response.status_code == 200
result = response.json()
# Version should be None or not present
assert result.get("version") is None
class TestVersionListing:
"""Tests for listing and retrieving versions."""
@pytest.mark.integration
def test_list_versions(self, integration_client, test_package):
"""Test listing all versions for a package."""
project, package = test_package
# Create multiple versions
for ver in ["1.0.0", "1.1.0", "2.0.0"]:
content = f"version {ver} content".encode()
files = {"file": (f"app-{ver}.tar.gz", io.BytesIO(content), "application/octet-stream")}
response = integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
data={"version": ver},
)
assert response.status_code == 200
# List versions
response = integration_client.get(
f"/api/v1/project/{project}/{package}/versions"
)
assert response.status_code == 200
data = response.json()
versions = [v["version"] for v in data.get("items", data)]
assert "1.0.0" in versions
assert "1.1.0" in versions
assert "2.0.0" in versions
@pytest.mark.integration
def test_get_specific_version(self, integration_client, test_package):
"""Test getting details for a specific version."""
project, package = test_package
content = b"specific version test"
expected_hash = compute_sha256(content)
# Create version
files = {"file": ("app-4.0.0.tar.gz", io.BytesIO(content), "application/octet-stream")}
integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
data={"version": "4.0.0"},
)
# Get version details
response = integration_client.get(
f"/api/v1/project/{project}/{package}/versions/4.0.0"
)
assert response.status_code == 200
data = response.json()
assert data["version"] == "4.0.0"
assert data["artifact_id"] == expected_hash
@pytest.mark.integration
def test_get_nonexistent_version_returns_404(self, integration_client, test_package):
"""Test getting nonexistent version returns 404."""
project, package = test_package
response = integration_client.get(
f"/api/v1/project/{project}/{package}/versions/99.99.99"
)
assert response.status_code == 404
class TestDownloadByVersion:
"""Tests for downloading artifacts by version."""
@pytest.mark.integration
def test_download_by_version_prefix(self, integration_client, test_package):
"""Test downloading artifact using version: prefix."""
project, package = test_package
content = b"download by version test"
expected_hash = compute_sha256(content)
# Upload with version
files = {"file": ("app.tar.gz", io.BytesIO(content), "application/octet-stream")}
integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
data={"version": "5.0.0"},
)
# Download by version prefix
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/version:5.0.0",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert response.content == content
@pytest.mark.integration
def test_download_nonexistent_version_returns_404(self, integration_client, test_package):
"""Test downloading nonexistent version returns 404."""
project, package = test_package
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/version:99.0.0"
)
assert response.status_code == 404
@pytest.mark.integration
def test_version_resolution_priority(self, integration_client, test_package):
"""Test that version: prefix explicitly resolves to version, not tag."""
project, package = test_package
version_content = b"this is the version content"
tag_content = b"this is the tag content"
# Create a version 6.0.0
files1 = {"file": ("app-v.tar.gz", io.BytesIO(version_content), "application/octet-stream")}
integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files1,
data={"version": "6.0.0"},
)
# Create a tag named "6.0.0" pointing to different content
files2 = {"file": ("app-t.tar.gz", io.BytesIO(tag_content), "application/octet-stream")}
integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files2,
data={"tag": "6.0.0"},
)
# Download with version: prefix should get version content
response = integration_client.get(
f"/api/v1/project/{project}/{package}/+/version:6.0.0",
params={"mode": "proxy"},
)
assert response.status_code == 200
assert response.content == version_content
# Download with tag: prefix should get tag content
response2 = integration_client.get(
f"/api/v1/project/{project}/{package}/+/tag:6.0.0",
params={"mode": "proxy"},
)
assert response2.status_code == 200
assert response2.content == tag_content
class TestVersionDeletion:
"""Tests for deleting versions."""
@pytest.mark.integration
def test_delete_version(self, integration_client, test_package):
"""Test deleting a version."""
project, package = test_package
content = b"delete version test"
# Create version
files = {"file": ("app.tar.gz", io.BytesIO(content), "application/octet-stream")}
integration_client.post(
f"/api/v1/project/{project}/{package}/upload",
files=files,
data={"version": "7.0.0"},
)
# Verify version exists
response = integration_client.get(
f"/api/v1/project/{project}/{package}/versions/7.0.0"
)
assert response.status_code == 200
# Delete version - returns 204 No Content on success
delete_response = integration_client.delete(
f"/api/v1/project/{project}/{package}/versions/7.0.0"
)
assert delete_response.status_code == 204
# Verify version no longer exists
response2 = integration_client.get(
f"/api/v1/project/{project}/{package}/versions/7.0.0"
)
assert response2.status_code == 404
@pytest.mark.integration
def test_delete_nonexistent_version_returns_404(self, integration_client, test_package):
"""Test deleting nonexistent version returns 404."""
project, package = test_package
response = integration_client.delete(
f"/api/v1/project/{project}/{package}/versions/99.0.0"
)
assert response.status_code == 404

View File

@@ -0,0 +1,412 @@
"""
Integration tests for version API endpoints.
Tests cover:
- Version creation via upload
- Version auto-detection from filename
- Version listing with pagination
- Version deletion
- Download by version ref
- ref_count behavior with version operations
"""
import pytest
from tests.factories import upload_test_file
class TestVersionCreation:
"""Tests for version creation during upload."""
@pytest.mark.integration
def test_upload_with_explicit_version(self, integration_client, test_package):
"""Test creating a version via explicit version parameter."""
project_name, package_name = test_package
result = upload_test_file(
integration_client,
project_name,
package_name,
b"version create test",
tag="latest",
version="1.0.0",
)
assert result["tag"] == "latest"
assert result["version"] == "1.0.0"
assert result["version_source"] == "explicit"
assert result["artifact_id"]
@pytest.mark.integration
def test_upload_with_version_auto_detect_from_tarball(
self, integration_client, test_package
):
"""Test version auto-detection from tarball filename pattern."""
project_name, package_name = test_package
result = upload_test_file(
integration_client,
project_name,
package_name,
b"auto version test",
filename="myapp-2.1.0.tar.gz",
)
assert result["version"] == "2.1.0"
# Tarball metadata extractor parses version from filename
assert result["version_source"] == "metadata"
@pytest.mark.integration
def test_upload_with_version_auto_detect_v_prefix(
self, integration_client, test_package
):
"""Test version auto-detection strips 'v' prefix from tarball filename."""
project_name, package_name = test_package
result = upload_test_file(
integration_client,
project_name,
package_name,
b"v prefix test",
filename="package-v3.0.0.tar.gz",
)
assert result["version"] == "3.0.0"
# Tarball metadata extractor parses version from filename
assert result["version_source"] == "metadata"
@pytest.mark.integration
def test_upload_duplicate_version_warning(self, integration_client, test_package):
"""Test that duplicate version during upload returns response without error."""
project_name, package_name = test_package
# Upload with version 1.0.0
upload_test_file(
integration_client,
project_name,
package_name,
b"first upload",
version="1.0.0",
)
# Upload different content with same version - should succeed but no new version
result = upload_test_file(
integration_client,
project_name,
package_name,
b"second upload different content",
version="1.0.0",
)
# Upload succeeds but version may not be set (duplicate)
assert result["artifact_id"]
class TestVersionCRUD:
"""Tests for version list, get, delete operations."""
@pytest.mark.integration
def test_list_versions(self, integration_client, test_package):
"""Test listing versions for a package."""
project_name, package_name = test_package
# Create some versions
upload_test_file(
integration_client,
project_name,
package_name,
b"v1 content",
version="1.0.0",
)
upload_test_file(
integration_client,
project_name,
package_name,
b"v2 content",
version="2.0.0",
)
response = integration_client.get(
f"/api/v1/project/{project_name}/{package_name}/versions"
)
assert response.status_code == 200
data = response.json()
assert "items" in data
assert "pagination" in data
versions = [v["version"] for v in data["items"]]
assert "1.0.0" in versions
assert "2.0.0" in versions
@pytest.mark.integration
def test_list_versions_with_artifact_info(self, integration_client, test_package):
"""Test that version list includes artifact metadata."""
project_name, package_name = test_package
upload_test_file(
integration_client,
project_name,
package_name,
b"version with info",
version="1.0.0",
tag="release",
)
response = integration_client.get(
f"/api/v1/project/{project_name}/{package_name}/versions"
)
assert response.status_code == 200
data = response.json()
assert len(data["items"]) >= 1
version_item = next(
(v for v in data["items"] if v["version"] == "1.0.0"), None
)
assert version_item is not None
assert "size" in version_item
assert "artifact_id" in version_item
assert "tags" in version_item
assert "release" in version_item["tags"]
@pytest.mark.integration
def test_get_version(self, integration_client, test_package):
"""Test getting a specific version."""
project_name, package_name = test_package
upload_result = upload_test_file(
integration_client,
project_name,
package_name,
b"get version test",
version="3.0.0",
)
response = integration_client.get(
f"/api/v1/project/{project_name}/{package_name}/versions/3.0.0"
)
assert response.status_code == 200
data = response.json()
assert data["version"] == "3.0.0"
assert data["artifact_id"] == upload_result["artifact_id"]
assert data["version_source"] == "explicit"
@pytest.mark.integration
def test_get_version_not_found(self, integration_client, test_package):
"""Test getting a non-existent version returns 404."""
project_name, package_name = test_package
response = integration_client.get(
f"/api/v1/project/{project_name}/{package_name}/versions/99.99.99"
)
assert response.status_code == 404
@pytest.mark.integration
def test_delete_version(self, integration_client, test_package):
"""Test deleting a version."""
project_name, package_name = test_package
upload_test_file(
integration_client,
project_name,
package_name,
b"delete version test",
version="4.0.0",
)
# Delete version
response = integration_client.delete(
f"/api/v1/project/{project_name}/{package_name}/versions/4.0.0"
)
assert response.status_code == 204
# Verify deleted
response = integration_client.get(
f"/api/v1/project/{project_name}/{package_name}/versions/4.0.0"
)
assert response.status_code == 404
class TestVersionDownload:
"""Tests for downloading artifacts by version reference."""
@pytest.mark.integration
def test_download_by_version_prefix(self, integration_client, test_package):
"""Test downloading an artifact using version: prefix."""
project_name, package_name = test_package
content = b"download by version test"
upload_test_file(
integration_client,
project_name,
package_name,
content,
version="5.0.0",
)
response = integration_client.get(
f"/api/v1/project/{project_name}/{package_name}/+/version:5.0.0",
follow_redirects=False,
)
# Should either redirect or return content
assert response.status_code in [200, 302, 307]
@pytest.mark.integration
def test_download_by_implicit_version(self, integration_client, test_package):
"""Test downloading an artifact using version number directly (no prefix)."""
project_name, package_name = test_package
content = b"implicit version download test"
upload_test_file(
integration_client,
project_name,
package_name,
content,
version="6.0.0",
)
response = integration_client.get(
f"/api/v1/project/{project_name}/{package_name}/+/6.0.0",
follow_redirects=False,
)
# Should resolve version first (before tag)
assert response.status_code in [200, 302, 307]
@pytest.mark.integration
def test_version_takes_precedence_over_tag(self, integration_client, test_package):
"""Test that version is checked before tag when resolving refs."""
project_name, package_name = test_package
# Upload with version "1.0"
version_result = upload_test_file(
integration_client,
project_name,
package_name,
b"version content",
version="1.0",
)
# Create a tag with the same name "1.0" pointing to different artifact
tag_result = upload_test_file(
integration_client,
project_name,
package_name,
b"tag content different",
tag="1.0",
)
# Download by "1.0" should resolve to version, not tag
# Since version:1.0 artifact was uploaded first
response = integration_client.get(
f"/api/v1/project/{project_name}/{package_name}/+/1.0",
follow_redirects=False,
)
assert response.status_code in [200, 302, 307]
class TestTagVersionEnrichment:
"""Tests for tag responses including version information."""
@pytest.mark.integration
def test_tag_response_includes_version(self, integration_client, test_package):
"""Test that tag responses include version of the artifact."""
project_name, package_name = test_package
# Upload with both version and tag
upload_test_file(
integration_client,
project_name,
package_name,
b"enriched tag test",
version="7.0.0",
tag="stable",
)
# Get tag and check version field
response = integration_client.get(
f"/api/v1/project/{project_name}/{package_name}/tags/stable"
)
assert response.status_code == 200
data = response.json()
assert data["name"] == "stable"
assert data["version"] == "7.0.0"
@pytest.mark.integration
def test_tag_list_includes_versions(self, integration_client, test_package):
"""Test that tag list responses include version for each tag."""
project_name, package_name = test_package
upload_test_file(
integration_client,
project_name,
package_name,
b"list version test",
version="8.0.0",
tag="latest",
)
response = integration_client.get(
f"/api/v1/project/{project_name}/{package_name}/tags"
)
assert response.status_code == 200
data = response.json()
tag_item = next((t for t in data["items"] if t["name"] == "latest"), None)
assert tag_item is not None
assert tag_item.get("version") == "8.0.0"
class TestVersionPagination:
"""Tests for version listing pagination and sorting."""
@pytest.mark.integration
def test_versions_pagination(self, integration_client, test_package):
"""Test version listing respects pagination."""
project_name, package_name = test_package
response = integration_client.get(
f"/api/v1/project/{project_name}/{package_name}/versions?limit=5"
)
assert response.status_code == 200
data = response.json()
assert "pagination" in data
assert data["pagination"]["limit"] == 5
@pytest.mark.integration
def test_versions_sorting(self, integration_client, test_package):
"""Test version listing can be sorted."""
project_name, package_name = test_package
# Create versions with different timestamps
upload_test_file(
integration_client,
project_name,
package_name,
b"sort test 1",
version="1.0.0",
)
upload_test_file(
integration_client,
project_name,
package_name,
b"sort test 2",
version="2.0.0",
)
# Test ascending sort
response = integration_client.get(
f"/api/v1/project/{project_name}/{package_name}/versions?sort=version&order=asc"
)
assert response.status_code == 200
data = response.json()
versions = [v["version"] for v in data["items"]]
# First version should be 1.0.0 when sorted ascending
if len(versions) >= 2:
assert versions.index("1.0.0") < versions.index("2.0.0")

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,294 @@
# Integrity Verification
Orchard uses content-addressable storage with SHA256 hashing to ensure artifact integrity. This document describes how integrity verification works and how to use it.
## How It Works
### Content-Addressable Storage
Orchard stores artifacts using their SHA256 hash as the unique identifier. This provides several benefits:
1. **Automatic deduplication**: Identical content is stored only once
2. **Built-in integrity**: The artifact ID *is* the content hash
3. **Tamper detection**: Any modification changes the hash, making corruption detectable
When you upload a file:
1. Orchard computes the SHA256 hash of the content
2. The hash becomes the artifact ID (64-character hex string)
3. The file is stored in S3 at `fruits/{hash[0:2]}/{hash[2:4]}/{hash}`
4. The hash and metadata are recorded in the database
### Hash Format
- Algorithm: SHA256
- Format: 64-character lowercase hexadecimal string
- Example: `dffd6021bb2bd5b0af676290809ec3a53191dd81c7f70a4b28688a362182986f`
## Client-Side Verification
### Before Upload
Compute the hash locally before uploading to verify the server received your content correctly:
```python
import hashlib
def compute_sha256(content: bytes) -> str:
return hashlib.sha256(content).hexdigest()
# Compute hash before upload
content = open("myfile.tar.gz", "rb").read()
local_hash = compute_sha256(content)
# Upload the file
response = requests.post(
f"{base_url}/api/v1/project/{project}/{package}/upload",
files={"file": ("myfile.tar.gz", content)},
)
result = response.json()
# Verify server computed the same hash
assert result["artifact_id"] == local_hash, "Hash mismatch!"
```
### Providing Expected Hash on Upload
You can provide the expected hash in the upload request. The server will reject the upload if the computed hash doesn't match:
```python
response = requests.post(
f"{base_url}/api/v1/project/{project}/{package}/upload",
files={"file": ("myfile.tar.gz", content)},
headers={"X-Checksum-SHA256": local_hash},
)
# Returns 422 if hash doesn't match
if response.status_code == 422:
print("Checksum mismatch - upload rejected")
```
### After Download
Verify downloaded content matches the expected hash using response headers:
```python
response = requests.get(
f"{base_url}/api/v1/project/{project}/{package}/+/{tag}",
params={"mode": "proxy"},
)
# Get expected hash from header
expected_hash = response.headers.get("X-Checksum-SHA256")
# Compute hash of downloaded content
actual_hash = compute_sha256(response.content)
# Verify
if actual_hash != expected_hash:
raise Exception(f"Integrity check failed! Expected {expected_hash}, got {actual_hash}")
```
### Response Headers for Verification
Download responses include multiple headers for verification:
| Header | Format | Description |
|--------|--------|-------------|
| `X-Checksum-SHA256` | Hex string | SHA256 hash (64 chars) |
| `ETag` | `"<hash>"` | SHA256 hash in quotes |
| `Digest` | `sha-256=<base64>` | RFC 3230 format (base64-encoded) |
| `Content-Length` | Integer | File size in bytes |
### Server-Side Verification on Download
Request server-side verification during download:
```bash
# Pre-verification: Server verifies before streaming (returns 500 if corrupt)
curl "${base_url}/api/v1/project/${project}/${package}/+/${tag}?mode=proxy&verify=true&verify_mode=pre"
# Stream verification: Server verifies while streaming (logs error if corrupt)
curl "${base_url}/api/v1/project/${project}/${package}/+/${tag}?mode=proxy&verify=true&verify_mode=stream"
```
The `X-Verified` header indicates whether server-side verification was performed:
- `X-Verified: true` - Content was verified by the server
## Server-Side Consistency Check
### Consistency Check Endpoint
Administrators can run a consistency check to verify all stored artifacts:
```bash
curl "${base_url}/api/v1/admin/consistency-check"
```
Response:
```json
{
"total_artifacts_checked": 1234,
"healthy": true,
"orphaned_s3_objects": 0,
"missing_s3_objects": 0,
"size_mismatches": 0,
"orphaned_s3_keys": [],
"missing_s3_keys": [],
"size_mismatch_artifacts": []
}
```
### What the Check Verifies
1. **Missing S3 objects**: Database records with no corresponding S3 object
2. **Orphaned S3 objects**: S3 objects with no database record
3. **Size mismatches**: S3 object size doesn't match database record
### Running Consistency Checks
**Manual check:**
```bash
# Check all artifacts
curl "${base_url}/api/v1/admin/consistency-check"
# Limit results (for large deployments)
curl "${base_url}/api/v1/admin/consistency-check?limit=100"
```
**Scheduled checks (recommended):**
Set up a cron job or Kubernetes CronJob to run periodic checks:
```yaml
# Kubernetes CronJob example
apiVersion: batch/v1
kind: CronJob
metadata:
name: orchard-consistency-check
spec:
schedule: "0 2 * * *" # Daily at 2 AM
jobTemplate:
spec:
template:
spec:
containers:
- name: check
image: curlimages/curl
command:
- /bin/sh
- -c
- |
response=$(curl -s "${ORCHARD_URL}/api/v1/admin/consistency-check")
healthy=$(echo "$response" | jq -r '.healthy')
if [ "$healthy" != "true" ]; then
echo "ALERT: Consistency check failed!"
echo "$response"
exit 1
fi
echo "Consistency check passed"
restartPolicy: OnFailure
```
## Recovery Procedures
### Corrupted Artifact (Size Mismatch)
If the consistency check reports size mismatches:
1. **Identify affected artifacts:**
```bash
curl "${base_url}/api/v1/admin/consistency-check" | jq '.size_mismatch_artifacts'
```
2. **Check if artifact can be re-uploaded:**
- If the original content is available, delete the corrupted artifact and re-upload
- The same content will produce the same artifact ID
3. **If original content is lost:**
- The artifact data is corrupted and cannot be recovered
- Delete the artifact record and notify affected users
- Consider restoring from backup if available
### Missing S3 Object
If database records exist but S3 objects are missing:
1. **Identify affected artifacts:**
```bash
curl "${base_url}/api/v1/admin/consistency-check" | jq '.missing_s3_keys'
```
2. **Check S3 bucket:**
- Verify the S3 bucket exists and is accessible
- Check S3 access logs for deletion events
- Check if objects were moved or lifecycle-deleted
3. **Recovery options:**
- Restore from S3 versioning (if enabled)
- Restore from backup
- Re-upload original content (if available)
- Delete orphaned database records
### Orphaned S3 Objects
If S3 objects exist without database records:
1. **Identify orphaned objects:**
```bash
curl "${base_url}/api/v1/admin/consistency-check" | jq '.orphaned_s3_keys'
```
2. **Investigate cause:**
- Upload interrupted before database commit?
- Database record deleted but S3 cleanup failed?
3. **Resolution:**
- If content is needed, create database record manually
- If content is not needed, delete the S3 object to reclaim storage
### Preventive Measures
1. **Enable S3 versioning** to recover from accidental deletions
2. **Regular backups** of both database and S3 bucket
3. **Scheduled consistency checks** to detect issues early
4. **Monitoring and alerting** on consistency check failures
5. **Audit logging** to track all artifact operations
## Verification in CI/CD
### Verifying Artifacts in Pipelines
```bash
#!/bin/bash
# Download and verify artifact in CI pipeline
ARTIFACT_URL="${ORCHARD_URL}/api/v1/project/${PROJECT}/${PACKAGE}/+/${TAG}"
# Download with verification headers
response=$(curl -s -D - "${ARTIFACT_URL}?mode=proxy" -o artifact.tar.gz)
expected_hash=$(echo "$response" | grep -i "X-Checksum-SHA256" | cut -d: -f2 | tr -d ' \r')
# Compute actual hash
actual_hash=$(sha256sum artifact.tar.gz | cut -d' ' -f1)
# Verify
if [ "$actual_hash" != "$expected_hash" ]; then
echo "ERROR: Integrity check failed!"
echo "Expected: $expected_hash"
echo "Actual: $actual_hash"
exit 1
fi
echo "Integrity verified: $actual_hash"
```
### Using Server-Side Verification
For critical deployments, use server-side pre-verification:
```bash
# Server verifies before streaming - returns 500 if corrupt
curl -f "${ARTIFACT_URL}?mode=proxy&verify=true&verify_mode=pre" -o artifact.tar.gz
```
This ensures the artifact is verified before any bytes are streamed to your pipeline.

View File

@@ -10,6 +10,7 @@ import ChangePasswordPage from './pages/ChangePasswordPage';
import APIKeysPage from './pages/APIKeysPage';
import AdminUsersPage from './pages/AdminUsersPage';
import AdminOIDCPage from './pages/AdminOIDCPage';
import ProjectSettingsPage from './pages/ProjectSettingsPage';
// Component that checks if user must change password
function RequirePasswordChange({ children }: { children: React.ReactNode }) {
@@ -45,6 +46,7 @@ function AppRoutes() {
<Route path="/admin/users" element={<AdminUsersPage />} />
<Route path="/admin/oidc" element={<AdminOIDCPage />} />
<Route path="/project/:projectName" element={<ProjectPage />} />
<Route path="/project/:projectName/settings" element={<ProjectSettingsPage />} />
<Route path="/project/:projectName/:packageName" element={<PackagePage />} />
</Routes>
</Layout>

View File

@@ -32,6 +32,10 @@ import {
OIDCConfig,
OIDCConfigUpdate,
OIDCStatus,
PackageVersion,
ArtifactDependenciesResponse,
ReverseDependenciesResponse,
DependencyResolutionResponse,
} from './types';
const API_BASE = '/api/v1';
@@ -170,6 +174,30 @@ export async function getProject(name: string): Promise<Project> {
return handleResponse<Project>(response);
}
export async function updateProject(
projectName: string,
data: { description?: string; is_public?: boolean }
): Promise<Project> {
const response = await fetch(`${API_BASE}/projects/${projectName}`, {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(data),
credentials: 'include',
});
return handleResponse<Project>(response);
}
export async function deleteProject(projectName: string): Promise<void> {
const response = await fetch(`${API_BASE}/projects/${projectName}`, {
method: 'DELETE',
credentials: 'include',
});
if (!response.ok) {
const error = await response.json().catch(() => ({ detail: 'Unknown error' }));
throw new Error(error.detail || `HTTP ${response.status}`);
}
}
// Package API
export async function listPackages(projectName: string, params: PackageListParams = {}): Promise<PaginatedResponse<Package>> {
const query = buildQueryString(params as Record<string, unknown>);
@@ -239,12 +267,21 @@ export async function listPackageArtifacts(
}
// Upload
export async function uploadArtifact(projectName: string, packageName: string, file: File, tag?: string): Promise<UploadResponse> {
export async function uploadArtifact(
projectName: string,
packageName: string,
file: File,
tag?: string,
version?: string
): Promise<UploadResponse> {
const formData = new FormData();
formData.append('file', file);
if (tag) {
formData.append('tag', tag);
}
if (version) {
formData.append('version', version);
}
const response = await fetch(`${API_BASE}/project/${projectName}/${packageName}/upload`, {
method: 'POST',
@@ -443,3 +480,85 @@ export function getOIDCLoginUrl(returnTo?: string): string {
const query = params.toString();
return `${API_BASE}/auth/oidc/login${query ? `?${query}` : ''}`;
}
// Version API
export async function listVersions(
projectName: string,
packageName: string,
params: ListParams = {}
): Promise<PaginatedResponse<PackageVersion>> {
const query = buildQueryString(params as Record<string, unknown>);
const response = await fetch(`${API_BASE}/project/${projectName}/${packageName}/versions${query}`);
return handleResponse<PaginatedResponse<PackageVersion>>(response);
}
export async function getVersion(
projectName: string,
packageName: string,
version: string
): Promise<PackageVersion> {
const response = await fetch(`${API_BASE}/project/${projectName}/${packageName}/versions/${version}`);
return handleResponse<PackageVersion>(response);
}
export async function deleteVersion(
projectName: string,
packageName: string,
version: string
): Promise<void> {
const response = await fetch(`${API_BASE}/project/${projectName}/${packageName}/versions/${version}`, {
method: 'DELETE',
credentials: 'include',
});
if (!response.ok) {
const error = await response.json().catch(() => ({ detail: 'Unknown error' }));
throw new Error(error.detail || `HTTP ${response.status}`);
}
}
// Dependency API
export async function getArtifactDependencies(artifactId: string): Promise<ArtifactDependenciesResponse> {
const response = await fetch(`${API_BASE}/artifact/${artifactId}/dependencies`);
return handleResponse<ArtifactDependenciesResponse>(response);
}
export async function getDependenciesByRef(
projectName: string,
packageName: string,
ref: string
): Promise<ArtifactDependenciesResponse> {
const response = await fetch(`${API_BASE}/project/${projectName}/${packageName}/+/${ref}/dependencies`);
return handleResponse<ArtifactDependenciesResponse>(response);
}
export async function getReverseDependencies(
projectName: string,
packageName: string,
params: { page?: number; limit?: number } = {}
): Promise<ReverseDependenciesResponse> {
const query = buildQueryString(params as Record<string, unknown>);
const response = await fetch(`${API_BASE}/project/${projectName}/${packageName}/reverse-dependencies${query}`);
return handleResponse<ReverseDependenciesResponse>(response);
}
export async function resolveDependencies(
projectName: string,
packageName: string,
ref: string
): Promise<DependencyResolutionResponse> {
const response = await fetch(`${API_BASE}/project/${projectName}/${packageName}/+/${ref}/resolve`);
return handleResponse<DependencyResolutionResponse>(response);
}
export async function getEnsureFile(
projectName: string,
packageName: string,
ref: string
): Promise<string> {
const response = await fetch(`${API_BASE}/project/${projectName}/${packageName}/+/${ref}/ensure`);
if (!response.ok) {
const error = await response.json().catch(() => ({ detail: 'Unknown error' }));
throw new ApiError(error.detail || `HTTP ${response.status}`, response.status);
}
return response.text();
}

View File

@@ -0,0 +1,338 @@
/* Dependency Graph Modal */
.dependency-graph-modal {
position: fixed;
top: 0;
left: 0;
right: 0;
bottom: 0;
background: rgba(0, 0, 0, 0.8);
display: flex;
align-items: center;
justify-content: center;
z-index: 1000;
padding: 24px;
}
.dependency-graph-content {
background: var(--bg-secondary);
border: 1px solid var(--border-primary);
border-radius: var(--radius-lg);
width: 100%;
max-width: 1200px;
height: 80vh;
display: flex;
flex-direction: column;
overflow: hidden;
}
.dependency-graph-header {
display: flex;
align-items: center;
gap: 16px;
padding: 16px 20px;
border-bottom: 1px solid var(--border-primary);
background: var(--bg-tertiary);
}
.dependency-graph-header h2 {
margin: 0;
font-size: 1.125rem;
font-weight: 600;
color: var(--text-primary);
}
.dependency-graph-info {
display: flex;
align-items: center;
gap: 12px;
flex: 1;
font-size: 0.875rem;
color: var(--text-secondary);
}
.graph-stats {
color: var(--text-muted);
font-size: 0.8125rem;
}
.close-btn {
background: transparent;
border: none;
color: var(--text-secondary);
cursor: pointer;
padding: 4px;
border-radius: var(--radius-sm);
display: flex;
align-items: center;
justify-content: center;
}
.close-btn:hover {
background: var(--bg-hover);
color: var(--text-primary);
}
.dependency-graph-toolbar {
display: flex;
align-items: center;
gap: 8px;
padding: 12px 20px;
border-bottom: 1px solid var(--border-primary);
background: var(--bg-secondary);
}
.zoom-level {
margin-left: auto;
font-size: 0.8125rem;
color: var(--text-muted);
font-family: 'JetBrains Mono', monospace;
}
.dependency-graph-container {
flex: 1;
overflow: hidden;
position: relative;
background:
linear-gradient(90deg, var(--border-primary) 1px, transparent 1px),
linear-gradient(var(--border-primary) 1px, transparent 1px);
background-size: 20px 20px;
background-position: center center;
}
.graph-canvas {
padding: 40px;
min-width: 100%;
min-height: 100%;
transform-origin: center center;
transition: transform 0.1s ease-out;
}
/* Graph Nodes */
.graph-node-container {
display: flex;
flex-direction: column;
align-items: flex-start;
}
.graph-node {
background: var(--bg-tertiary);
border: 2px solid var(--border-primary);
border-radius: var(--radius-md);
padding: 12px 16px;
min-width: 200px;
cursor: pointer;
transition: all var(--transition-fast);
position: relative;
}
.graph-node:hover {
border-color: var(--accent-primary);
box-shadow: 0 4px 12px rgba(16, 185, 129, 0.2);
}
.graph-node--root {
background: linear-gradient(135deg, rgba(16, 185, 129, 0.15) 0%, rgba(5, 150, 105, 0.15) 100%);
border-color: var(--accent-primary);
}
.graph-node--hovered {
transform: scale(1.02);
}
.graph-node__header {
display: flex;
align-items: center;
gap: 8px;
margin-bottom: 4px;
}
.graph-node__name {
font-weight: 600;
color: var(--accent-primary);
font-family: 'JetBrains Mono', monospace;
font-size: 0.875rem;
}
.graph-node__toggle {
background: var(--bg-hover);
border: 1px solid var(--border-primary);
border-radius: 4px;
width: 20px;
height: 20px;
display: flex;
align-items: center;
justify-content: center;
cursor: pointer;
font-size: 0.875rem;
color: var(--text-secondary);
font-weight: 600;
margin-left: auto;
}
.graph-node__toggle:hover {
background: var(--bg-tertiary);
color: var(--text-primary);
}
.graph-node__details {
display: flex;
align-items: center;
gap: 12px;
font-size: 0.75rem;
color: var(--text-muted);
}
.graph-node__version {
font-family: 'JetBrains Mono', monospace;
color: var(--text-secondary);
}
.graph-node__size {
color: var(--text-muted);
}
/* Graph Children / Tree Structure */
.graph-children {
display: flex;
padding-left: 24px;
margin-top: 8px;
position: relative;
}
.graph-connector {
position: absolute;
left: 12px;
top: 0;
bottom: 50%;
width: 12px;
border-left: 2px solid var(--border-primary);
border-bottom: 2px solid var(--border-primary);
border-bottom-left-radius: 8px;
}
.graph-children-list {
display: flex;
flex-direction: column;
gap: 8px;
position: relative;
}
.graph-children-list::before {
content: '';
position: absolute;
left: -12px;
top: 20px;
bottom: 20px;
border-left: 2px solid var(--border-primary);
}
.graph-children-list > .graph-node-container {
position: relative;
}
.graph-children-list > .graph-node-container::before {
content: '';
position: absolute;
left: -12px;
top: 20px;
width: 12px;
border-top: 2px solid var(--border-primary);
}
/* Loading, Error, Empty States */
.graph-loading,
.graph-error,
.graph-empty {
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
height: 100%;
gap: 16px;
color: var(--text-muted);
}
.graph-loading .spinner {
width: 32px;
height: 32px;
border: 3px solid var(--border-primary);
border-top-color: var(--accent-primary);
border-radius: 50%;
animation: spin 1s linear infinite;
}
@keyframes spin {
to { transform: rotate(360deg); }
}
.graph-error {
color: var(--error-color, #ef4444);
}
.graph-error svg {
opacity: 0.6;
}
.graph-error p {
max-width: 400px;
text-align: center;
line-height: 1.5;
}
/* Tooltip */
.graph-tooltip {
position: fixed;
bottom: 24px;
left: 50%;
transform: translateX(-50%);
background: var(--bg-tertiary);
border: 1px solid var(--border-primary);
border-radius: var(--radius-md);
padding: 12px 16px;
font-size: 0.8125rem;
box-shadow: 0 8px 24px rgba(0, 0, 0, 0.4);
z-index: 1001;
}
.graph-tooltip strong {
display: block;
color: var(--accent-primary);
font-family: 'JetBrains Mono', monospace;
margin-bottom: 4px;
}
.graph-tooltip div {
color: var(--text-secondary);
margin-top: 2px;
}
.tooltip-hint {
margin-top: 8px;
padding-top: 8px;
border-top: 1px solid var(--border-primary);
color: var(--text-muted);
font-size: 0.75rem;
}
/* Responsive */
@media (max-width: 768px) {
.dependency-graph-modal {
padding: 0;
}
.dependency-graph-content {
height: 100vh;
border-radius: 0;
max-width: none;
}
.dependency-graph-header {
flex-wrap: wrap;
}
.dependency-graph-info {
flex-basis: 100%;
order: 3;
margin-top: 8px;
}
}

View File

@@ -0,0 +1,323 @@
import { useState, useEffect, useCallback, useRef } from 'react';
import { useNavigate } from 'react-router-dom';
import { ResolvedArtifact, DependencyResolutionResponse, Dependency } from '../types';
import { resolveDependencies, getArtifactDependencies } from '../api';
import './DependencyGraph.css';
interface DependencyGraphProps {
projectName: string;
packageName: string;
tagName: string;
onClose: () => void;
}
interface GraphNode {
id: string;
project: string;
package: string;
version: string | null;
size: number;
depth: number;
children: GraphNode[];
isRoot?: boolean;
}
function formatBytes(bytes: number): string {
if (bytes === 0) return '0 B';
const k = 1024;
const sizes = ['B', 'KB', 'MB', 'GB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return parseFloat((bytes / Math.pow(k, i)).toFixed(1)) + ' ' + sizes[i];
}
function DependencyGraph({ projectName, packageName, tagName, onClose }: DependencyGraphProps) {
const navigate = useNavigate();
const containerRef = useRef<HTMLDivElement>(null);
const [loading, setLoading] = useState(true);
const [error, setError] = useState<string | null>(null);
const [resolution, setResolution] = useState<DependencyResolutionResponse | null>(null);
const [graphRoot, setGraphRoot] = useState<GraphNode | null>(null);
const [hoveredNode, setHoveredNode] = useState<GraphNode | null>(null);
const [zoom, setZoom] = useState(1);
const [pan, setPan] = useState({ x: 0, y: 0 });
const [isDragging, setIsDragging] = useState(false);
const [dragStart, setDragStart] = useState({ x: 0, y: 0 });
const [collapsedNodes, setCollapsedNodes] = useState<Set<string>>(new Set());
// Build graph structure from resolution data
const buildGraph = useCallback(async (resolutionData: DependencyResolutionResponse) => {
const artifactMap = new Map<string, ResolvedArtifact>();
resolutionData.resolved.forEach(artifact => {
artifactMap.set(artifact.artifact_id, artifact);
});
// Fetch dependencies for each artifact to build the tree
const depsMap = new Map<string, Dependency[]>();
for (const artifact of resolutionData.resolved) {
try {
const deps = await getArtifactDependencies(artifact.artifact_id);
depsMap.set(artifact.artifact_id, deps.dependencies);
} catch {
depsMap.set(artifact.artifact_id, []);
}
}
// Find the root artifact (the requested one)
const rootArtifact = resolutionData.resolved.find(
a => a.project === resolutionData.requested.project &&
a.package === resolutionData.requested.package
);
if (!rootArtifact) {
return null;
}
// Build tree recursively
const visited = new Set<string>();
const buildNode = (artifact: ResolvedArtifact, depth: number): GraphNode => {
const nodeId = `${artifact.project}/${artifact.package}`;
visited.add(artifact.artifact_id);
const deps = depsMap.get(artifact.artifact_id) || [];
const children: GraphNode[] = [];
for (const dep of deps) {
// Find the resolved artifact for this dependency
const childArtifact = resolutionData.resolved.find(
a => a.project === dep.project && a.package === dep.package
);
if (childArtifact && !visited.has(childArtifact.artifact_id)) {
children.push(buildNode(childArtifact, depth + 1));
}
}
return {
id: nodeId,
project: artifact.project,
package: artifact.package,
version: artifact.version || artifact.tag,
size: artifact.size,
depth,
children,
isRoot: depth === 0,
};
};
return buildNode(rootArtifact, 0);
}, []);
useEffect(() => {
async function loadData() {
setLoading(true);
setError(null);
try {
const result = await resolveDependencies(projectName, packageName, tagName);
setResolution(result);
const graph = await buildGraph(result);
setGraphRoot(graph);
} catch (err) {
if (err instanceof Error) {
// Check if it's a resolution error
try {
const errorData = JSON.parse(err.message);
if (errorData.error === 'circular_dependency') {
setError(`Circular dependency detected: ${errorData.cycle?.join(' → ')}`);
} else if (errorData.error === 'dependency_conflict') {
setError(`Dependency conflict: ${errorData.message}`);
} else {
setError(err.message);
}
} catch {
setError(err.message);
}
} else {
setError('Failed to load dependency graph');
}
} finally {
setLoading(false);
}
}
loadData();
}, [projectName, packageName, tagName, buildGraph]);
const handleNodeClick = (node: GraphNode) => {
navigate(`/project/${node.project}/${node.package}`);
onClose();
};
const handleNodeToggle = (node: GraphNode, e: React.MouseEvent) => {
e.stopPropagation();
setCollapsedNodes(prev => {
const next = new Set(prev);
if (next.has(node.id)) {
next.delete(node.id);
} else {
next.add(node.id);
}
return next;
});
};
const handleWheel = (e: React.WheelEvent) => {
e.preventDefault();
const delta = e.deltaY > 0 ? -0.1 : 0.1;
setZoom(z => Math.max(0.25, Math.min(2, z + delta)));
};
const handleMouseDown = (e: React.MouseEvent) => {
if (e.target === containerRef.current || (e.target as HTMLElement).classList.contains('graph-canvas')) {
setIsDragging(true);
setDragStart({ x: e.clientX - pan.x, y: e.clientY - pan.y });
}
};
const handleMouseMove = (e: React.MouseEvent) => {
if (isDragging) {
setPan({ x: e.clientX - dragStart.x, y: e.clientY - dragStart.y });
}
};
const handleMouseUp = () => {
setIsDragging(false);
};
const resetView = () => {
setZoom(1);
setPan({ x: 0, y: 0 });
};
const renderNode = (node: GraphNode, index: number = 0): JSX.Element => {
const isCollapsed = collapsedNodes.has(node.id);
const hasChildren = node.children.length > 0;
return (
<div key={`${node.id}-${index}`} className="graph-node-container">
<div
className={`graph-node ${node.isRoot ? 'graph-node--root' : ''} ${hoveredNode?.id === node.id ? 'graph-node--hovered' : ''}`}
onClick={() => handleNodeClick(node)}
onMouseEnter={() => setHoveredNode(node)}
onMouseLeave={() => setHoveredNode(null)}
>
<div className="graph-node__header">
<span className="graph-node__name">{node.project}/{node.package}</span>
{hasChildren && (
<button
className="graph-node__toggle"
onClick={(e) => handleNodeToggle(node, e)}
title={isCollapsed ? 'Expand' : 'Collapse'}
>
{isCollapsed ? '+' : '-'}
</button>
)}
</div>
<div className="graph-node__details">
{node.version && <span className="graph-node__version">@ {node.version}</span>}
<span className="graph-node__size">{formatBytes(node.size)}</span>
</div>
</div>
{hasChildren && !isCollapsed && (
<div className="graph-children">
<div className="graph-connector"></div>
<div className="graph-children-list">
{node.children.map((child, i) => renderNode(child, i))}
</div>
</div>
)}
</div>
);
};
return (
<div className="dependency-graph-modal" onClick={onClose}>
<div className="dependency-graph-content" onClick={e => e.stopPropagation()}>
<div className="dependency-graph-header">
<h2>Dependency Graph</h2>
<div className="dependency-graph-info">
<span>{projectName}/{packageName} @ {tagName}</span>
{resolution && (
<span className="graph-stats">
{resolution.artifact_count} packages {formatBytes(resolution.total_size)} total
</span>
)}
</div>
<button className="close-btn" onClick={onClose} title="Close">
<svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2">
<line x1="18" y1="6" x2="6" y2="18"></line>
<line x1="6" y1="6" x2="18" y2="18"></line>
</svg>
</button>
</div>
<div className="dependency-graph-toolbar">
<button className="btn btn-secondary btn-small" onClick={() => setZoom(z => Math.min(2, z + 0.25))}>
Zoom In
</button>
<button className="btn btn-secondary btn-small" onClick={() => setZoom(z => Math.max(0.25, z - 0.25))}>
Zoom Out
</button>
<button className="btn btn-secondary btn-small" onClick={resetView}>
Reset View
</button>
<span className="zoom-level">{Math.round(zoom * 100)}%</span>
</div>
<div
ref={containerRef}
className="dependency-graph-container"
onWheel={handleWheel}
onMouseDown={handleMouseDown}
onMouseMove={handleMouseMove}
onMouseUp={handleMouseUp}
onMouseLeave={handleMouseUp}
>
{loading ? (
<div className="graph-loading">
<div className="spinner"></div>
<span>Resolving dependencies...</span>
</div>
) : error ? (
<div className="graph-error">
<svg width="48" height="48" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2">
<circle cx="12" cy="12" r="10"></circle>
<line x1="12" y1="8" x2="12" y2="12"></line>
<line x1="12" y1="16" x2="12.01" y2="16"></line>
</svg>
<p>{error}</p>
</div>
) : graphRoot ? (
<div
className="graph-canvas"
style={{
transform: `translate(${pan.x}px, ${pan.y}px) scale(${zoom})`,
cursor: isDragging ? 'grabbing' : 'grab',
}}
>
{renderNode(graphRoot)}
</div>
) : (
<div className="graph-empty">No dependencies to display</div>
)}
</div>
{hoveredNode && (
<div className="graph-tooltip">
<strong>{hoveredNode.project}/{hoveredNode.package}</strong>
{hoveredNode.version && <div>Version: {hoveredNode.version}</div>}
<div>Size: {formatBytes(hoveredNode.size)}</div>
<div className="tooltip-hint">Click to navigate</div>
</div>
)}
</div>
</div>
);
}
export default DependencyGraph;

View File

@@ -193,7 +193,6 @@ function Layout({ children }: LayoutProps) {
</div>
<div className="footer-links">
<a href="/docs">Documentation</a>
<a href="/api/v1">API</a>
</div>
</div>
</footer>

View File

@@ -358,6 +358,12 @@
gap: 4px;
}
.page-header__actions {
display: flex;
align-items: center;
gap: 12px;
}
/* Package card styles */
.package-card__header {
display: flex;

View File

@@ -127,6 +127,12 @@ h2 {
font-size: 0.75rem;
}
/* Action buttons in table */
.action-buttons {
display: flex;
gap: 8px;
}
/* Download by Artifact ID Section */
.download-by-id-section {
margin-top: 32px;
@@ -324,6 +330,86 @@ tr:hover .copy-btn {
color: var(--text-muted);
}
/* Version badge */
.version-badge {
font-family: 'JetBrains Mono', 'Fira Code', 'Consolas', monospace;
font-size: 0.8125rem;
color: var(--accent-primary);
background: rgba(16, 185, 129, 0.1);
padding: 2px 8px;
border-radius: var(--radius-sm);
}
/* Create Tag Section */
.create-tag-section {
margin-top: 32px;
background: var(--bg-secondary);
}
.create-tag-section h3 {
margin-bottom: 4px;
color: var(--text-primary);
font-size: 1rem;
font-weight: 600;
}
.section-description {
color: var(--text-muted);
font-size: 0.875rem;
margin-bottom: 16px;
}
.create-tag-form .form-row {
display: flex;
gap: 12px;
align-items: flex-end;
flex-wrap: wrap;
}
.create-tag-form .form-group {
flex: 1;
min-width: 150px;
}
.create-tag-form .form-group--wide {
flex: 2;
min-width: 300px;
}
.create-tag-form .form-group label {
display: block;
margin-bottom: 6px;
font-size: 0.75rem;
font-weight: 500;
color: var(--text-secondary);
text-transform: uppercase;
letter-spacing: 0.05em;
}
.create-tag-form .form-group input {
width: 100%;
padding: 10px 14px;
background: var(--bg-tertiary);
border: 1px solid var(--border-primary);
border-radius: var(--radius-md);
color: var(--text-primary);
font-size: 0.875rem;
}
.create-tag-form .form-group input:focus {
outline: none;
border-color: var(--accent-primary);
}
.create-tag-form .form-group input:disabled {
opacity: 0.6;
cursor: not-allowed;
}
.create-tag-form button {
flex-shrink: 0;
}
/* Created cell */
.created-cell {
display: flex;
@@ -344,6 +430,340 @@ tr:hover .copy-btn {
white-space: nowrap;
}
/* Dependencies Section */
.dependencies-section {
margin-top: 32px;
background: var(--bg-secondary);
}
.dependencies-header {
display: flex;
align-items: center;
justify-content: space-between;
margin-bottom: 12px;
}
.dependencies-header h3 {
margin: 0;
color: var(--text-primary);
font-size: 1rem;
font-weight: 600;
}
.dependencies-controls {
display: flex;
align-items: center;
gap: 8px;
}
.dependencies-controls .btn {
display: inline-flex;
align-items: center;
}
.dependencies-tag-select {
margin-bottom: 16px;
}
.tag-selector {
padding: 8px 12px;
background: var(--bg-tertiary);
border: 1px solid var(--border-primary);
border-radius: var(--radius-md);
color: var(--text-primary);
font-size: 0.875rem;
cursor: pointer;
min-width: 200px;
}
.tag-selector:focus {
outline: none;
border-color: var(--accent-primary);
}
.deps-loading {
color: var(--text-muted);
font-size: 0.875rem;
padding: 16px 0;
}
.deps-error {
color: var(--error-color, #ef4444);
font-size: 0.875rem;
padding: 12px 16px;
background: rgba(239, 68, 68, 0.1);
border-radius: var(--radius-md);
}
.deps-empty {
color: var(--text-muted);
font-size: 0.875rem;
padding: 16px 0;
}
.deps-summary {
color: var(--text-secondary);
font-size: 0.875rem;
margin-bottom: 12px;
}
.deps-summary strong {
color: var(--accent-primary);
}
.deps-items {
list-style: none;
margin: 0;
padding: 0;
display: flex;
flex-direction: column;
gap: 8px;
}
.dep-item {
display: flex;
align-items: center;
gap: 12px;
padding: 12px 16px;
background: var(--bg-tertiary);
border-radius: var(--radius-md);
border: 1px solid var(--border-primary);
}
.dep-link {
color: var(--accent-primary);
font-weight: 500;
text-decoration: none;
font-family: 'JetBrains Mono', 'Fira Code', 'Consolas', monospace;
font-size: 0.875rem;
}
.dep-link:hover {
text-decoration: underline;
}
.dep-constraint {
color: var(--text-muted);
font-size: 0.8125rem;
font-family: 'JetBrains Mono', 'Fira Code', 'Consolas', monospace;
}
.dep-status {
margin-left: auto;
font-size: 0.875rem;
font-weight: 600;
}
.dep-status--ok {
color: var(--success-color, #10b981);
}
.dep-status--missing {
color: var(--warning-color, #f59e0b);
}
/* Tag name link in table */
.tag-name-link {
color: var(--accent-primary);
transition: opacity var(--transition-fast);
}
.tag-name-link:hover {
opacity: 0.8;
}
.tag-name-link.selected {
text-decoration: underline;
}
/* Used By (Reverse Dependencies) Section */
.used-by-section {
margin-top: 32px;
background: var(--bg-secondary);
}
.used-by-section h3 {
margin-bottom: 16px;
color: var(--text-primary);
font-size: 1rem;
font-weight: 600;
}
.reverse-dep-item {
display: flex;
align-items: center;
gap: 12px;
flex-wrap: wrap;
}
.dep-version {
color: var(--accent-primary);
font-family: 'JetBrains Mono', 'Fira Code', 'Consolas', monospace;
font-size: 0.8125rem;
background: rgba(16, 185, 129, 0.1);
padding: 2px 8px;
border-radius: var(--radius-sm);
}
.dep-requires {
color: var(--text-muted);
font-size: 0.8125rem;
font-family: 'JetBrains Mono', 'Fira Code', 'Consolas', monospace;
margin-left: auto;
}
.reverse-deps-pagination {
display: flex;
align-items: center;
justify-content: center;
gap: 16px;
margin-top: 16px;
padding-top: 16px;
border-top: 1px solid var(--border-primary);
}
.pagination-info {
color: var(--text-secondary);
font-size: 0.875rem;
}
/* Ensure File Modal */
.modal-overlay {
position: fixed;
top: 0;
left: 0;
right: 0;
bottom: 0;
background: rgba(0, 0, 0, 0.7);
display: flex;
align-items: center;
justify-content: center;
z-index: 1000;
padding: 20px;
}
.ensure-file-modal {
background: var(--bg-secondary);
border: 1px solid var(--border-primary);
border-radius: var(--radius-lg);
max-width: 700px;
width: 100%;
max-height: 80vh;
display: flex;
flex-direction: column;
box-shadow: 0 20px 50px rgba(0, 0, 0, 0.5);
}
.ensure-file-header {
display: flex;
align-items: center;
justify-content: space-between;
padding: 16px 20px;
border-bottom: 1px solid var(--border-primary);
}
.ensure-file-header h3 {
margin: 0;
color: var(--text-primary);
font-size: 1rem;
font-weight: 600;
}
.ensure-file-actions {
display: flex;
align-items: center;
gap: 8px;
}
.ensure-file-actions .copy-btn {
opacity: 1;
width: 32px;
height: 32px;
}
.modal-close {
display: flex;
align-items: center;
justify-content: center;
width: 32px;
height: 32px;
padding: 0;
background: transparent;
border: none;
border-radius: var(--radius-sm);
color: var(--text-muted);
cursor: pointer;
transition: all var(--transition-fast);
}
.modal-close:hover {
background: var(--bg-hover);
color: var(--text-primary);
}
.ensure-file-content {
flex: 1;
overflow: auto;
padding: 20px;
}
.ensure-file-loading {
color: var(--text-muted);
text-align: center;
padding: 40px 20px;
}
.ensure-file-error {
color: var(--error-color, #ef4444);
padding: 16px;
background: rgba(239, 68, 68, 0.1);
border-radius: var(--radius-md);
}
.ensure-file-empty {
color: var(--text-muted);
text-align: center;
padding: 40px 20px;
font-style: italic;
}
.ensure-file-yaml {
margin: 0;
padding: 16px;
background: #0d0d0f;
border: 1px solid var(--border-primary);
border-radius: var(--radius-md);
overflow-x: auto;
}
.ensure-file-yaml code {
font-family: 'JetBrains Mono', 'Fira Code', 'Consolas', monospace;
font-size: 0.8125rem;
color: #e2e8f0;
white-space: pre;
}
.ensure-file-footer {
padding: 16px 20px;
border-top: 1px solid var(--border-primary);
background: var(--bg-tertiary);
border-radius: 0 0 var(--radius-lg) var(--radius-lg);
}
.ensure-file-hint {
margin: 0;
color: var(--text-muted);
font-size: 0.8125rem;
}
.ensure-file-hint code {
background: rgba(0, 0, 0, 0.2);
padding: 2px 6px;
border-radius: var(--radius-sm);
font-family: 'JetBrains Mono', 'Fira Code', 'Consolas', monospace;
color: var(--accent-primary);
}
/* Responsive adjustments */
@media (max-width: 768px) {
.upload-form {
@@ -359,4 +779,18 @@ tr:hover .copy-btn {
flex-wrap: wrap;
gap: 12px;
}
.dependencies-header {
flex-direction: column;
align-items: flex-start;
gap: 12px;
}
.tag-selector {
width: 100%;
}
.ensure-file-modal {
max-height: 90vh;
}
}

View File

@@ -1,7 +1,7 @@
import { useState, useEffect, useCallback } from 'react';
import { useParams, useSearchParams, useNavigate, useLocation } from 'react-router-dom';
import { TagDetail, Package, PaginatedResponse, AccessLevel } from '../types';
import { listTags, getDownloadUrl, getPackage, getMyProjectAccess, UnauthorizedError, ForbiddenError } from '../api';
import { useParams, useSearchParams, useNavigate, useLocation, Link } from 'react-router-dom';
import { TagDetail, Package, PaginatedResponse, AccessLevel, Dependency, DependentInfo } from '../types';
import { listTags, getDownloadUrl, getPackage, getMyProjectAccess, createTag, getArtifactDependencies, getReverseDependencies, getEnsureFile, UnauthorizedError, ForbiddenError } from '../api';
import { Breadcrumb } from '../components/Breadcrumb';
import { Badge } from '../components/Badge';
import { SearchInput } from '../components/SearchInput';
@@ -10,6 +10,7 @@ import { DataTable } from '../components/DataTable';
import { Pagination } from '../components/Pagination';
import { DragDropUpload, UploadResult } from '../components/DragDropUpload';
import { useAuth } from '../contexts/AuthContext';
import DependencyGraph from '../components/DependencyGraph';
import './Home.css';
import './PackagePage.css';
@@ -64,6 +65,33 @@ function PackagePage() {
const [uploadSuccess, setUploadSuccess] = useState<string | null>(null);
const [artifactIdInput, setArtifactIdInput] = useState('');
const [accessLevel, setAccessLevel] = useState<AccessLevel | null>(null);
const [createTagName, setCreateTagName] = useState('');
const [createTagArtifactId, setCreateTagArtifactId] = useState('');
const [createTagLoading, setCreateTagLoading] = useState(false);
// Dependencies state
const [selectedTag, setSelectedTag] = useState<TagDetail | null>(null);
const [dependencies, setDependencies] = useState<Dependency[]>([]);
const [depsLoading, setDepsLoading] = useState(false);
const [depsError, setDepsError] = useState<string | null>(null);
// Reverse dependencies state
const [reverseDeps, setReverseDeps] = useState<DependentInfo[]>([]);
const [reverseDepsLoading, setReverseDepsLoading] = useState(false);
const [reverseDepsError, setReverseDepsError] = useState<string | null>(null);
const [reverseDepsPage, setReverseDepsPage] = useState(1);
const [reverseDepsTotal, setReverseDepsTotal] = useState(0);
const [reverseDepsHasMore, setReverseDepsHasMore] = useState(false);
// Dependency graph modal state
const [showGraph, setShowGraph] = useState(false);
// Ensure file modal state
const [showEnsureFile, setShowEnsureFile] = useState(false);
const [ensureFileContent, setEnsureFileContent] = useState<string | null>(null);
const [ensureFileLoading, setEnsureFileLoading] = useState(false);
const [ensureFileError, setEnsureFileError] = useState<string | null>(null);
const [ensureFileTagName, setEnsureFileTagName] = useState<string | null>(null);
// Derived permissions
const canWrite = accessLevel === 'write' || accessLevel === 'admin';
@@ -125,6 +153,98 @@ function PackagePage() {
loadData();
}, [loadData]);
// Auto-select tag when tags are loaded (prefer version from URL, then first tag)
// Re-run when package changes to pick up new tags
useEffect(() => {
if (tagsData?.items && tagsData.items.length > 0) {
const versionParam = searchParams.get('version');
if (versionParam) {
// Find tag matching the version parameter
const matchingTag = tagsData.items.find(t => t.version === versionParam);
if (matchingTag) {
setSelectedTag(matchingTag);
setDependencies([]);
return;
}
}
// Fall back to first tag
setSelectedTag(tagsData.items[0]);
setDependencies([]);
}
}, [tagsData, searchParams, projectName, packageName]);
// Fetch dependencies when selected tag changes
const fetchDependencies = useCallback(async (artifactId: string) => {
setDepsLoading(true);
setDepsError(null);
try {
const result = await getArtifactDependencies(artifactId);
setDependencies(result.dependencies);
} catch (err) {
setDepsError(err instanceof Error ? err.message : 'Failed to load dependencies');
setDependencies([]);
} finally {
setDepsLoading(false);
}
}, []);
useEffect(() => {
if (selectedTag) {
fetchDependencies(selectedTag.artifact_id);
}
}, [selectedTag, fetchDependencies]);
// Fetch reverse dependencies
const fetchReverseDeps = useCallback(async (pageNum: number = 1) => {
if (!projectName || !packageName) return;
setReverseDepsLoading(true);
setReverseDepsError(null);
try {
const result = await getReverseDependencies(projectName, packageName, { page: pageNum, limit: 10 });
setReverseDeps(result.dependents);
setReverseDepsTotal(result.pagination.total);
setReverseDepsHasMore(result.pagination.has_more);
setReverseDepsPage(pageNum);
} catch (err) {
setReverseDepsError(err instanceof Error ? err.message : 'Failed to load reverse dependencies');
setReverseDeps([]);
} finally {
setReverseDepsLoading(false);
}
}, [projectName, packageName]);
useEffect(() => {
if (projectName && packageName && !loading) {
fetchReverseDeps(1);
}
}, [projectName, packageName, loading, fetchReverseDeps]);
// Fetch ensure file for a specific tag
const fetchEnsureFileForTag = useCallback(async (tagName: string) => {
if (!projectName || !packageName) return;
setEnsureFileTagName(tagName);
setEnsureFileLoading(true);
setEnsureFileError(null);
try {
const content = await getEnsureFile(projectName, packageName, tagName);
setEnsureFileContent(content);
setShowEnsureFile(true);
} catch (err) {
setEnsureFileError(err instanceof Error ? err.message : 'Failed to load ensure file');
setShowEnsureFile(true);
} finally {
setEnsureFileLoading(false);
}
}, [projectName, packageName]);
// Fetch ensure file for selected tag
const fetchEnsureFile = useCallback(async () => {
if (!selectedTag) return;
fetchEnsureFileForTag(selectedTag.name);
}, [selectedTag, fetchEnsureFileForTag]);
// Keyboard navigation - go back with backspace
useEffect(() => {
const handleKeyDown = (e: KeyboardEvent) => {
@@ -154,6 +274,30 @@ function PackagePage() {
setError(errorMsg);
}, []);
const handleCreateTag = async (e: React.FormEvent) => {
e.preventDefault();
if (!createTagName.trim() || createTagArtifactId.length !== 64) return;
setCreateTagLoading(true);
setError(null);
try {
await createTag(projectName!, packageName!, {
name: createTagName.trim(),
artifact_id: createTagArtifactId,
});
setUploadSuccess(`Tag "${createTagName}" created successfully!`);
setCreateTagName('');
setCreateTagArtifactId('');
loadData();
setTimeout(() => setUploadSuccess(null), 5000);
} catch (err) {
setError(err instanceof Error ? err.message : 'Failed to create tag');
} finally {
setCreateTagLoading(false);
}
};
const handleSearchChange = (value: string) => {
updateParams({ search: value, page: '1' });
};
@@ -175,12 +319,31 @@ function PackagePage() {
const tags = tagsData?.items || [];
const pagination = tagsData?.pagination;
const handleTagSelect = (tag: TagDetail) => {
setSelectedTag(tag);
};
const columns = [
{
key: 'name',
header: 'Tag',
sortable: true,
render: (t: TagDetail) => <strong>{t.name}</strong>,
render: (t: TagDetail) => (
<strong
className={`tag-name-link ${selectedTag?.id === t.id ? 'selected' : ''}`}
onClick={() => handleTagSelect(t)}
style={{ cursor: 'pointer' }}
>
{t.name}
</strong>
),
},
{
key: 'version',
header: 'Version',
render: (t: TagDetail) => (
<span className="version-badge">{t.version || '-'}</span>
),
},
{
key: 'artifact_id',
@@ -227,13 +390,22 @@ function PackagePage() {
key: 'actions',
header: 'Actions',
render: (t: TagDetail) => (
<a
href={getDownloadUrl(projectName!, packageName!, t.name)}
className="btn btn-secondary btn-small"
download
>
Download
</a>
<div className="action-buttons">
<button
className="btn btn-secondary btn-small"
onClick={() => fetchEnsureFileForTag(t.name)}
title="View orchard.ensure file"
>
Ensure
</button>
<a
href={getDownloadUrl(projectName!, packageName!, t.name)}
className="btn btn-secondary btn-small"
download
>
Download
</a>
</div>
),
},
];
@@ -405,6 +577,166 @@ function PackagePage() {
/>
)}
{/* Dependencies Section */}
{tags.length > 0 && (
<div className="dependencies-section card">
<div className="dependencies-header">
<h3>Dependencies</h3>
<div className="dependencies-controls">
{selectedTag && (
<>
<button
className="btn btn-secondary btn-small"
onClick={fetchEnsureFile}
disabled={ensureFileLoading}
title="View orchard.ensure file"
>
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" style={{ marginRight: '6px' }}>
<path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z"></path>
<polyline points="14 2 14 8 20 8"></polyline>
<line x1="16" y1="13" x2="8" y2="13"></line>
<line x1="16" y1="17" x2="8" y2="17"></line>
<polyline points="10 9 9 9 8 9"></polyline>
</svg>
{ensureFileLoading ? 'Loading...' : 'View Ensure File'}
</button>
<button
className="btn btn-secondary btn-small"
onClick={() => setShowGraph(true)}
title="View full dependency tree"
>
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" style={{ marginRight: '6px' }}>
<circle cx="12" cy="12" r="3"></circle>
<circle cx="4" cy="4" r="2"></circle>
<circle cx="20" cy="4" r="2"></circle>
<circle cx="4" cy="20" r="2"></circle>
<circle cx="20" cy="20" r="2"></circle>
<line x1="9.5" y1="9.5" x2="5.5" y2="5.5"></line>
<line x1="14.5" y1="9.5" x2="18.5" y2="5.5"></line>
<line x1="9.5" y1="14.5" x2="5.5" y2="18.5"></line>
<line x1="14.5" y1="14.5" x2="18.5" y2="18.5"></line>
</svg>
View Graph
</button>
</>
)}
</div>
</div>
<div className="dependencies-tag-select">
{selectedTag && (
<select
className="tag-selector"
value={selectedTag.id}
onChange={(e) => {
const tag = tags.find(t => t.id === e.target.value);
if (tag) setSelectedTag(tag);
}}
>
{tags.map(t => (
<option key={t.id} value={t.id}>
{t.name}{t.version ? ` (${t.version})` : ''}
</option>
))}
</select>
)}
</div>
{depsLoading ? (
<div className="deps-loading">Loading dependencies...</div>
) : depsError ? (
<div className="deps-error">{depsError}</div>
) : dependencies.length === 0 ? (
<div className="deps-empty">
{selectedTag ? (
<span><strong>{selectedTag.name}</strong> has no dependencies</span>
) : (
<span>No dependencies</span>
)}
</div>
) : (
<div className="deps-list">
<div className="deps-summary">
<strong>{selectedTag?.name}</strong> has {dependencies.length} {dependencies.length === 1 ? 'dependency' : 'dependencies'}:
</div>
<ul className="deps-items">
{dependencies.map((dep) => (
<li key={dep.id} className="dep-item">
<Link
to={`/project/${dep.project}/${dep.package}`}
className="dep-link"
>
{dep.project}/{dep.package}
</Link>
<span className="dep-constraint">
@ {dep.version || dep.tag}
</span>
<span className="dep-status dep-status--ok" title="Package exists">
&#10003;
</span>
</li>
))}
</ul>
</div>
)}
</div>
)}
{/* Used By (Reverse Dependencies) Section */}
<div className="used-by-section card">
<h3>Used By</h3>
{reverseDepsLoading ? (
<div className="deps-loading">Loading reverse dependencies...</div>
) : reverseDepsError ? (
<div className="deps-error">{reverseDepsError}</div>
) : reverseDeps.length === 0 ? (
<div className="deps-empty">No packages depend on this package</div>
) : (
<div className="reverse-deps-list">
<div className="deps-summary">
{reverseDepsTotal} {reverseDepsTotal === 1 ? 'package depends' : 'packages depend'} on this:
</div>
<ul className="deps-items">
{reverseDeps.map((dep) => (
<li key={dep.artifact_id} className="dep-item reverse-dep-item">
<Link
to={`/project/${dep.project}/${dep.package}${dep.version ? `?version=${dep.version}` : ''}`}
className="dep-link"
>
{dep.project}/{dep.package}
{dep.version && (
<span className="dep-version">v{dep.version}</span>
)}
</Link>
<span className="dep-requires">
requires @ {dep.constraint_value}
</span>
</li>
))}
</ul>
{(reverseDepsHasMore || reverseDepsPage > 1) && (
<div className="reverse-deps-pagination">
<button
className="btn btn-secondary btn-small"
onClick={() => fetchReverseDeps(reverseDepsPage - 1)}
disabled={reverseDepsPage <= 1 || reverseDepsLoading}
>
Previous
</button>
<span className="pagination-info">Page {reverseDepsPage}</span>
<button
className="btn btn-secondary btn-small"
onClick={() => fetchReverseDeps(reverseDepsPage + 1)}
disabled={!reverseDepsHasMore || reverseDepsLoading}
>
Next
</button>
</div>
)}
</div>
)}
</div>
<div className="download-by-id-section card">
<h3>Download by Artifact ID</h3>
<div className="download-by-id-form">
@@ -433,6 +765,50 @@ function PackagePage() {
)}
</div>
{user && canWrite && (
<div className="create-tag-section card">
<h3>Create / Update Tag</h3>
<p className="section-description">Point a tag at any existing artifact by its ID</p>
<form onSubmit={handleCreateTag} className="create-tag-form">
<div className="form-row">
<div className="form-group">
<label htmlFor="create-tag-name">Tag Name</label>
<input
id="create-tag-name"
type="text"
value={createTagName}
onChange={(e) => setCreateTagName(e.target.value)}
placeholder="latest, stable, v1.0.0..."
disabled={createTagLoading}
/>
</div>
<div className="form-group form-group--wide">
<label htmlFor="create-tag-artifact">Artifact ID</label>
<input
id="create-tag-artifact"
type="text"
value={createTagArtifactId}
onChange={(e) => setCreateTagArtifactId(e.target.value.toLowerCase().replace(/[^a-f0-9]/g, '').slice(0, 64))}
placeholder="SHA256 hash (64 hex characters)"
className="artifact-id-input"
disabled={createTagLoading}
/>
</div>
<button
type="submit"
className="btn btn-primary"
disabled={createTagLoading || !createTagName.trim() || createTagArtifactId.length !== 64}
>
{createTagLoading ? 'Creating...' : 'Create Tag'}
</button>
</div>
{createTagArtifactId.length > 0 && createTagArtifactId.length !== 64 && (
<p className="validation-hint">Artifact ID must be exactly 64 hex characters ({createTagArtifactId.length}/64)</p>
)}
</form>
</div>
)}
<div className="usage-section card">
<h3>Usage</h3>
<p>Download artifacts using:</p>
@@ -444,6 +820,58 @@ function PackagePage() {
<code>curl -O {window.location.origin}/api/v1/project/{projectName}/{packageName}/+/v1.0.0</code>
</pre>
</div>
{/* Dependency Graph Modal */}
{showGraph && selectedTag && (
<DependencyGraph
projectName={projectName!}
packageName={packageName!}
tagName={selectedTag.name}
onClose={() => setShowGraph(false)}
/>
)}
{/* Ensure File Modal */}
{showEnsureFile && (
<div className="modal-overlay" onClick={() => setShowEnsureFile(false)}>
<div className="ensure-file-modal" onClick={(e) => e.stopPropagation()}>
<div className="ensure-file-header">
<h3>orchard.ensure for {ensureFileTagName}</h3>
<div className="ensure-file-actions">
{ensureFileContent && (
<CopyButton text={ensureFileContent} />
)}
<button
className="modal-close"
onClick={() => setShowEnsureFile(false)}
title="Close"
>
<svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2">
<line x1="18" y1="6" x2="6" y2="18"></line>
<line x1="6" y1="6" x2="18" y2="18"></line>
</svg>
</button>
</div>
</div>
<div className="ensure-file-content">
{ensureFileLoading ? (
<div className="ensure-file-loading">Loading...</div>
) : ensureFileError ? (
<div className="ensure-file-error">{ensureFileError}</div>
) : ensureFileContent ? (
<pre className="ensure-file-yaml"><code>{ensureFileContent}</code></pre>
) : (
<div className="ensure-file-empty">No dependencies defined for this artifact.</div>
)}
</div>
<div className="ensure-file-footer">
<p className="ensure-file-hint">
Save this as <code>orchard.ensure</code> in your project root to declare dependencies.
</p>
</div>
</div>
</div>
)}
</div>
);
}

View File

@@ -8,7 +8,6 @@ import { DataTable } from '../components/DataTable';
import { SearchInput } from '../components/SearchInput';
import { FilterChip, FilterChipGroup } from '../components/FilterChip';
import { Pagination } from '../components/Pagination';
import { AccessManagement } from '../components/AccessManagement';
import { useAuth } from '../contexts/AuthContext';
import './Home.css';
@@ -211,15 +210,30 @@ function ProjectPage() {
<span className="meta-item">by {project.created_by}</span>
</div>
</div>
{canWrite ? (
<button className="btn btn-primary" onClick={() => setShowForm(!showForm)}>
{showForm ? 'Cancel' : '+ New Package'}
</button>
) : user ? (
<span className="text-muted" title="You have read-only access to this project">
Read-only access
</span>
) : null}
<div className="page-header__actions">
{canAdmin && (
<button
className="btn btn-secondary"
onClick={() => navigate(`/project/${projectName}/settings`)}
title="Project Settings"
>
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round">
<circle cx="12" cy="12" r="3" />
<path d="M19.4 15a1.65 1.65 0 0 0 .33 1.82l.06.06a2 2 0 0 1 0 2.83 2 2 0 0 1-2.83 0l-.06-.06a1.65 1.65 0 0 0-1.82-.33 1.65 1.65 0 0 0-1 1.51V21a2 2 0 0 1-2 2 2 2 0 0 1-2-2v-.09A1.65 1.65 0 0 0 9 19.4a1.65 1.65 0 0 0-1.82.33l-.06.06a2 2 0 0 1-2.83 0 2 2 0 0 1 0-2.83l.06-.06a1.65 1.65 0 0 0 .33-1.82 1.65 1.65 0 0 0-1.51-1H3a2 2 0 0 1-2-2 2 2 0 0 1 2-2h.09A1.65 1.65 0 0 0 4.6 9a1.65 1.65 0 0 0-.33-1.82l-.06-.06a2 2 0 0 1 0-2.83 2 2 0 0 1 2.83 0l.06.06a1.65 1.65 0 0 0 1.82.33H9a1.65 1.65 0 0 0 1-1.51V3a2 2 0 0 1 2-2 2 2 0 0 1 2 2v.09a1.65 1.65 0 0 0 1 1.51 1.65 1.65 0 0 0 1.82-.33l.06-.06a2 2 0 0 1 2.83 0 2 2 0 0 1 0 2.83l-.06.06a1.65 1.65 0 0 0-.33 1.82V9a1.65 1.65 0 0 0 1.51 1H21a2 2 0 0 1 2 2 2 2 0 0 1-2 2h-.09a1.65 1.65 0 0 0-1.51 1z" />
</svg>
Settings
</button>
)}
{canWrite ? (
<button className="btn btn-primary" onClick={() => setShowForm(!showForm)}>
{showForm ? 'Cancel' : '+ New Package'}
</button>
) : user ? (
<span className="text-muted" title="You have read-only access to this project">
Read-only access
</span>
) : null}
</div>
</div>
{error && <div className="error-message">{error}</div>}
@@ -371,10 +385,6 @@ function ProjectPage() {
onPageChange={handlePageChange}
/>
)}
{canAdmin && projectName && (
<AccessManagement projectName={projectName} />
)}
</div>
);
}

View File

@@ -0,0 +1,476 @@
.project-settings-page {
max-width: 900px;
margin: 0 auto;
}
.project-settings-header {
margin-bottom: 32px;
}
.project-settings-header h1 {
font-size: 1.75rem;
font-weight: 600;
color: var(--text-primary);
margin-bottom: 8px;
letter-spacing: -0.02em;
}
.project-settings-subtitle {
color: var(--text-tertiary);
font-size: 0.9375rem;
}
.project-settings-loading {
display: flex;
align-items: center;
justify-content: center;
gap: 12px;
padding: 64px 24px;
color: var(--text-tertiary);
font-size: 0.9375rem;
}
.project-settings-spinner {
width: 20px;
height: 20px;
border: 2px solid var(--border-secondary);
border-top-color: var(--accent-primary);
border-radius: 50%;
animation: project-settings-spin 0.6s linear infinite;
}
@keyframes project-settings-spin {
to {
transform: rotate(360deg);
}
}
.project-settings-error {
display: flex;
align-items: center;
gap: 10px;
background: var(--error-bg);
border: 1px solid rgba(239, 68, 68, 0.2);
color: var(--error);
padding: 12px 16px;
border-radius: var(--radius-md);
margin-bottom: 24px;
font-size: 0.875rem;
}
.project-settings-success {
display: flex;
align-items: center;
gap: 10px;
background: var(--success-bg);
border: 1px solid rgba(34, 197, 94, 0.2);
color: var(--success);
padding: 12px 16px;
border-radius: var(--radius-md);
margin-bottom: 24px;
font-size: 0.875rem;
animation: project-settings-fade-in 0.2s ease;
}
@keyframes project-settings-fade-in {
from {
opacity: 0;
transform: translateY(-8px);
}
to {
opacity: 1;
transform: translateY(0);
}
}
.project-settings-section {
background: var(--bg-secondary);
border: 1px solid var(--border-primary);
border-radius: var(--radius-lg);
padding: 24px;
margin-bottom: 24px;
}
.project-settings-section h2 {
font-size: 1.125rem;
font-weight: 600;
color: var(--text-primary);
margin-bottom: 20px;
padding-bottom: 16px;
border-bottom: 1px solid var(--border-primary);
}
.project-settings-form {
display: flex;
flex-direction: column;
gap: 16px;
}
.project-settings-form-group {
display: flex;
flex-direction: column;
gap: 6px;
}
.project-settings-form-group label {
font-size: 0.8125rem;
font-weight: 500;
color: var(--text-secondary);
}
.project-settings-form-group textarea,
.project-settings-form-group input[type="text"] {
padding: 12px 14px;
background: var(--bg-tertiary);
border: 1px solid var(--border-primary);
border-radius: var(--radius-md);
font-size: 0.875rem;
color: var(--text-primary);
transition: all var(--transition-fast);
font-family: inherit;
resize: vertical;
}
.project-settings-form-group textarea {
min-height: 100px;
}
.project-settings-form-group textarea::placeholder,
.project-settings-form-group input::placeholder {
color: var(--text-muted);
}
.project-settings-form-group textarea:hover:not(:disabled),
.project-settings-form-group input:hover:not(:disabled) {
border-color: var(--border-secondary);
background: var(--bg-elevated);
}
.project-settings-form-group textarea:focus,
.project-settings-form-group input:focus {
outline: none;
border-color: var(--accent-primary);
box-shadow: 0 0 0 3px rgba(16, 185, 129, 0.15);
background: var(--bg-elevated);
}
.project-settings-form-group textarea:disabled,
.project-settings-form-group input:disabled {
opacity: 0.6;
cursor: not-allowed;
}
.project-settings-checkbox-group {
flex-direction: row;
align-items: center;
}
.project-settings-checkbox-label {
display: flex;
align-items: center;
gap: 10px;
cursor: pointer;
font-size: 0.875rem;
font-weight: 400;
color: var(--text-secondary);
user-select: none;
}
.project-settings-checkbox-label input[type="checkbox"] {
position: absolute;
opacity: 0;
width: 0;
height: 0;
}
.project-settings-checkbox-custom {
width: 18px;
height: 18px;
background: var(--bg-tertiary);
border: 1px solid var(--border-secondary);
border-radius: var(--radius-sm);
transition: all var(--transition-fast);
position: relative;
flex-shrink: 0;
}
.project-settings-checkbox-label input[type="checkbox"]:checked + .project-settings-checkbox-custom {
background: var(--accent-primary);
border-color: var(--accent-primary);
}
.project-settings-checkbox-label input[type="checkbox"]:checked + .project-settings-checkbox-custom::after {
content: '';
position: absolute;
left: 5px;
top: 2px;
width: 5px;
height: 9px;
border: solid white;
border-width: 0 2px 2px 0;
transform: rotate(45deg);
}
.project-settings-checkbox-label input[type="checkbox"]:focus + .project-settings-checkbox-custom {
box-shadow: 0 0 0 3px rgba(16, 185, 129, 0.15);
}
.project-settings-checkbox-label:hover .project-settings-checkbox-custom {
border-color: var(--accent-primary);
}
.project-settings-form-actions {
display: flex;
justify-content: flex-end;
gap: 12px;
margin-top: 8px;
}
.project-settings-save-button {
display: flex;
align-items: center;
justify-content: center;
gap: 8px;
padding: 10px 18px;
background: var(--accent-gradient);
border: none;
border-radius: var(--radius-md);
font-size: 0.875rem;
font-weight: 500;
color: white;
cursor: pointer;
transition: all var(--transition-fast);
min-width: 120px;
}
.project-settings-save-button:hover:not(:disabled) {
transform: translateY(-1px);
box-shadow: var(--shadow-sm), 0 0 20px rgba(16, 185, 129, 0.2);
}
.project-settings-save-button:disabled {
opacity: 0.5;
cursor: not-allowed;
transform: none;
}
.project-settings-button-spinner {
width: 14px;
height: 14px;
border: 2px solid rgba(255, 255, 255, 0.3);
border-top-color: white;
border-radius: 50%;
animation: project-settings-spin 0.6s linear infinite;
}
/* Danger Zone */
.project-settings-danger-zone {
background: var(--bg-secondary);
border: 1px solid rgba(239, 68, 68, 0.3);
border-radius: var(--radius-lg);
padding: 24px;
margin-bottom: 24px;
}
.project-settings-danger-zone h2 {
font-size: 1.125rem;
font-weight: 600;
color: var(--error);
margin-bottom: 20px;
padding-bottom: 16px;
border-bottom: 1px solid rgba(239, 68, 68, 0.2);
}
.project-settings-danger-item {
display: flex;
justify-content: space-between;
align-items: flex-start;
gap: 24px;
}
.project-settings-danger-info h3 {
font-size: 0.9375rem;
font-weight: 600;
color: var(--text-primary);
margin-bottom: 4px;
}
.project-settings-danger-info p {
color: var(--text-tertiary);
font-size: 0.8125rem;
max-width: 400px;
}
.project-settings-delete-button {
padding: 10px 18px;
background: transparent;
border: 1px solid rgba(239, 68, 68, 0.3);
border-radius: var(--radius-md);
font-size: 0.875rem;
font-weight: 500;
color: var(--error);
cursor: pointer;
transition: all var(--transition-fast);
flex-shrink: 0;
}
.project-settings-delete-button:hover:not(:disabled) {
background: var(--error-bg);
border-color: rgba(239, 68, 68, 0.5);
}
.project-settings-delete-button:disabled {
opacity: 0.5;
cursor: not-allowed;
}
/* Delete Confirmation */
.project-settings-delete-confirm {
margin-top: 20px;
padding-top: 20px;
border-top: 1px solid rgba(239, 68, 68, 0.2);
animation: project-settings-fade-in 0.2s ease;
}
.project-settings-delete-confirm p {
color: var(--text-secondary);
font-size: 0.875rem;
margin-bottom: 12px;
}
.project-settings-delete-confirm strong {
color: var(--text-primary);
font-family: 'JetBrains Mono', 'Fira Code', 'SF Mono', Monaco, monospace;
background: var(--bg-tertiary);
padding: 2px 6px;
border-radius: var(--radius-sm);
}
.project-settings-delete-confirm-input {
width: 100%;
padding: 12px 14px;
background: var(--bg-tertiary);
border: 1px solid var(--border-primary);
border-radius: var(--radius-md);
font-size: 0.875rem;
color: var(--text-primary);
transition: all var(--transition-fast);
margin-bottom: 16px;
}
.project-settings-delete-confirm-input:focus {
outline: none;
border-color: var(--error);
box-shadow: 0 0 0 3px rgba(239, 68, 68, 0.15);
}
.project-settings-delete-confirm-input::placeholder {
color: var(--text-muted);
}
.project-settings-delete-confirm-actions {
display: flex;
gap: 12px;
}
.project-settings-confirm-delete-button {
display: flex;
align-items: center;
justify-content: center;
gap: 8px;
padding: 10px 18px;
background: var(--error);
border: none;
border-radius: var(--radius-md);
font-size: 0.875rem;
font-weight: 500;
color: white;
cursor: pointer;
transition: all var(--transition-fast);
min-width: 120px;
}
.project-settings-confirm-delete-button:hover:not(:disabled) {
opacity: 0.9;
}
.project-settings-confirm-delete-button:disabled {
opacity: 0.5;
cursor: not-allowed;
}
.project-settings-cancel-button {
padding: 10px 18px;
background: transparent;
border: 1px solid var(--border-primary);
border-radius: var(--radius-md);
font-size: 0.875rem;
font-weight: 500;
color: var(--text-secondary);
cursor: pointer;
transition: all var(--transition-fast);
}
.project-settings-cancel-button:hover:not(:disabled) {
background: var(--bg-hover);
border-color: var(--border-secondary);
color: var(--text-primary);
}
.project-settings-cancel-button:disabled {
opacity: 0.5;
cursor: not-allowed;
}
.project-settings-delete-spinner {
width: 14px;
height: 14px;
border: 2px solid rgba(255, 255, 255, 0.3);
border-top-color: white;
border-radius: 50%;
animation: project-settings-spin 0.6s linear infinite;
}
/* Access denied */
.project-settings-access-denied {
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
padding: 80px 24px;
text-align: center;
}
.project-settings-access-denied h2 {
font-size: 1.5rem;
font-weight: 600;
color: var(--text-primary);
margin-bottom: 12px;
}
.project-settings-access-denied p {
color: var(--text-tertiary);
font-size: 0.9375rem;
max-width: 400px;
}
/* Responsive */
@media (max-width: 768px) {
.project-settings-danger-item {
flex-direction: column;
gap: 16px;
}
.project-settings-delete-button {
align-self: flex-start;
}
.project-settings-delete-confirm-actions {
flex-direction: column;
}
.project-settings-confirm-delete-button,
.project-settings-cancel-button {
width: 100%;
}
}

View File

@@ -0,0 +1,308 @@
import { useState, useEffect, useCallback } from 'react';
import { useParams, useNavigate } from 'react-router-dom';
import { Project } from '../types';
import {
getProject,
updateProject,
deleteProject,
getMyProjectAccess,
UnauthorizedError,
ForbiddenError,
} from '../api';
import { Breadcrumb } from '../components/Breadcrumb';
import { AccessManagement } from '../components/AccessManagement';
import { useAuth } from '../contexts/AuthContext';
import './ProjectSettingsPage.css';
function ProjectSettingsPage() {
const { projectName } = useParams<{ projectName: string }>();
const navigate = useNavigate();
const { user } = useAuth();
const [project, setProject] = useState<Project | null>(null);
const [loading, setLoading] = useState(true);
const [error, setError] = useState<string | null>(null);
const [success, setSuccess] = useState<string | null>(null);
const [accessDenied, setAccessDenied] = useState(false);
const [canAdmin, setCanAdmin] = useState(false);
// General settings form state
const [description, setDescription] = useState('');
const [isPublic, setIsPublic] = useState(false);
const [saving, setSaving] = useState(false);
// Delete confirmation state
const [showDeleteConfirm, setShowDeleteConfirm] = useState(false);
const [deleteConfirmText, setDeleteConfirmText] = useState('');
const [deleting, setDeleting] = useState(false);
const loadData = useCallback(async () => {
if (!projectName) return;
try {
setLoading(true);
setAccessDenied(false);
const [projectData, accessResult] = await Promise.all([
getProject(projectName),
getMyProjectAccess(projectName),
]);
setProject(projectData);
setDescription(projectData.description || '');
setIsPublic(projectData.is_public);
const hasAdminAccess = accessResult.access_level === 'admin';
setCanAdmin(hasAdminAccess);
if (!hasAdminAccess) {
setAccessDenied(true);
}
setError(null);
} catch (err) {
if (err instanceof UnauthorizedError) {
navigate('/login', { state: { from: `/project/${projectName}/settings` } });
return;
}
if (err instanceof ForbiddenError) {
setAccessDenied(true);
setLoading(false);
return;
}
setError(err instanceof Error ? err.message : 'Failed to load project');
} finally {
setLoading(false);
}
}, [projectName, navigate]);
useEffect(() => {
loadData();
}, [loadData]);
const handleSaveSettings = async (e: React.FormEvent) => {
e.preventDefault();
if (!projectName) return;
try {
setSaving(true);
setError(null);
const updatedProject = await updateProject(projectName, {
description: description || undefined,
is_public: isPublic,
});
setProject(updatedProject);
setSuccess('Settings saved successfully');
setTimeout(() => setSuccess(null), 3000);
} catch (err) {
setError(err instanceof Error ? err.message : 'Failed to save settings');
} finally {
setSaving(false);
}
};
const handleDeleteProject = async () => {
if (!projectName || deleteConfirmText !== projectName) return;
try {
setDeleting(true);
setError(null);
await deleteProject(projectName);
navigate('/');
} catch (err) {
setError(err instanceof Error ? err.message : 'Failed to delete project');
setDeleting(false);
}
};
const handleCancelDelete = () => {
setShowDeleteConfirm(false);
setDeleteConfirmText('');
};
if (loading) {
return (
<div className="project-settings-page">
<Breadcrumb
items={[
{ label: 'Projects', href: '/' },
{ label: projectName || '', href: `/project/${projectName}` },
{ label: 'Settings' },
]}
/>
<div className="project-settings-loading">
<div className="project-settings-spinner" />
<span>Loading...</span>
</div>
</div>
);
}
if (accessDenied || !canAdmin) {
return (
<div className="project-settings-page">
<Breadcrumb
items={[
{ label: 'Projects', href: '/' },
{ label: projectName || '', href: `/project/${projectName}` },
{ label: 'Settings' },
]}
/>
<div className="project-settings-access-denied">
<h2>Access Denied</h2>
<p>You must be a project admin to access settings.</p>
{!user && (
<p style={{ marginTop: '16px' }}>
<a href="/login" className="btn btn-primary">
Sign in
</a>
</p>
)}
</div>
</div>
);
}
if (!project) {
return (
<div className="project-settings-page">
<Breadcrumb
items={[
{ label: 'Projects', href: '/' },
{ label: projectName || '' },
]}
/>
<div className="project-settings-error">Project not found</div>
</div>
);
}
return (
<div className="project-settings-page">
<Breadcrumb
items={[
{ label: 'Projects', href: '/' },
{ label: project.name, href: `/project/${project.name}` },
{ label: 'Settings' },
]}
/>
<div className="project-settings-header">
<h1>Project Settings</h1>
<p className="project-settings-subtitle">Manage settings for {project.name}</p>
</div>
{error && <div className="project-settings-error">{error}</div>}
{success && <div className="project-settings-success">{success}</div>}
{/* General Settings Section */}
<div className="project-settings-section">
<h2>General</h2>
<form className="project-settings-form" onSubmit={handleSaveSettings}>
<div className="project-settings-form-group">
<label htmlFor="description">Description</label>
<textarea
id="description"
value={description}
onChange={(e) => setDescription(e.target.value)}
placeholder="Describe your project..."
disabled={saving}
/>
</div>
<div className="project-settings-form-group project-settings-checkbox-group">
<label className="project-settings-checkbox-label">
<input
type="checkbox"
checked={isPublic}
onChange={(e) => setIsPublic(e.target.checked)}
disabled={saving}
/>
<span className="project-settings-checkbox-custom" />
<span>Public project (visible to everyone)</span>
</label>
</div>
<div className="project-settings-form-actions">
<button type="submit" className="project-settings-save-button" disabled={saving}>
{saving ? (
<>
<span className="project-settings-button-spinner" />
Saving...
</>
) : (
'Save Changes'
)}
</button>
</div>
</form>
</div>
{/* Access Management Section */}
<AccessManagement projectName={projectName!} />
{/* Danger Zone Section */}
<div className="project-settings-danger-zone">
<h2>Danger Zone</h2>
<div className="project-settings-danger-item">
<div className="project-settings-danger-info">
<h3>Delete this project</h3>
<p>
Once you delete a project, there is no going back. This will permanently delete the
project, all packages, artifacts, and tags.
</p>
</div>
{!showDeleteConfirm && (
<button
className="project-settings-delete-button"
onClick={() => setShowDeleteConfirm(true)}
disabled={deleting}
>
Delete Project
</button>
)}
</div>
{showDeleteConfirm && (
<div className="project-settings-delete-confirm">
<p>
Type <strong>{project.name}</strong> to confirm deletion:
</p>
<input
type="text"
className="project-settings-delete-confirm-input"
value={deleteConfirmText}
onChange={(e) => setDeleteConfirmText(e.target.value)}
placeholder={project.name}
disabled={deleting}
autoFocus
/>
<div className="project-settings-delete-confirm-actions">
<button
className="project-settings-confirm-delete-button"
onClick={handleDeleteProject}
disabled={deleting || deleteConfirmText !== project.name}
>
{deleting ? (
<>
<span className="project-settings-delete-spinner" />
Deleting...
</>
) : (
'Yes, delete this project'
)}
</button>
<button
className="project-settings-cancel-button"
onClick={handleCancelDelete}
disabled={deleting}
>
Cancel
</button>
</div>
</div>
)}
</div>
</div>
);
}
export default ProjectSettingsPage;

View File

@@ -63,6 +63,22 @@ export interface TagDetail extends Tag {
artifact_original_name: string | null;
artifact_created_at: string;
artifact_format_metadata: Record<string, unknown> | null;
version: string | null;
}
export interface PackageVersion {
id: string;
package_id: string;
artifact_id: string;
version: string;
version_source: string | null;
created_at: string;
created_by: string;
// Enriched fields from joins
size?: number;
content_type?: string | null;
original_name?: string | null;
tags?: string[];
}
export interface ArtifactTagInfo {
@@ -122,6 +138,8 @@ export interface UploadResponse {
project: string;
package: string;
tag: string | null;
version: string | null;
version_source: string | null;
}
// Global search types
@@ -355,3 +373,77 @@ export interface OIDCStatus {
enabled: boolean;
issuer_url?: string;
}
// Dependency types
export interface Dependency {
id: string;
artifact_id: string;
project: string;
package: string;
version: string | null;
tag: string | null;
created_at: string;
}
export interface ArtifactDependenciesResponse {
artifact_id: string;
dependencies: Dependency[];
}
export interface DependentInfo {
artifact_id: string;
project: string;
package: string;
version: string | null;
constraint_type: 'version' | 'tag';
constraint_value: string;
}
export interface ReverseDependenciesResponse {
project: string;
package: string;
dependents: DependentInfo[];
pagination: {
page: number;
limit: number;
total: number;
total_pages: number;
has_more: boolean;
};
}
// Dependency Resolution types
export interface ResolvedArtifact {
artifact_id: string;
project: string;
package: string;
version: string | null;
tag: string | null;
size: number;
download_url: string;
}
export interface DependencyResolutionResponse {
requested: {
project: string;
package: string;
ref: string;
};
resolved: ResolvedArtifact[];
total_size: number;
artifact_count: number;
}
export interface DependencyResolutionError {
error: 'circular_dependency' | 'dependency_conflict' | 'not_found';
message: string;
cycle?: string[];
conflicts?: Array<{
project: string;
package: string;
requirements: Array<{
version: string;
required_by: Array<{ path: string }>;
}>;
}>;
}

View File

@@ -77,6 +77,8 @@ PostgreSQL secret name
{{- define "orchard.postgresql.secretName" -}}
{{- if .Values.orchard.database.existingSecret }}
{{- .Values.orchard.database.existingSecret }}
{{- else if and .Values.orchard.database.secretsManager .Values.orchard.database.secretsManager.enabled }}
{{- printf "%s-db-credentials" (include "orchard.fullname" .) }}
{{- else if .Values.postgresql.enabled }}
{{- printf "%s-postgresql" .Release.Name }}
{{- else }}
@@ -90,6 +92,8 @@ PostgreSQL password key in secret
{{- define "orchard.postgresql.passwordKey" -}}
{{- if .Values.orchard.database.existingSecret -}}
{{- .Values.orchard.database.existingSecretPasswordKey -}}
{{- else if and .Values.orchard.database.secretsManager .Values.orchard.database.secretsManager.enabled -}}
password
{{- else if .Values.postgresql.enabled -}}
password
{{- else -}}

View File

@@ -69,6 +69,8 @@ spec:
containerPort: {{ .Values.orchard.server.port }}
protocol: TCP
env:
- name: ORCHARD_ENV
value: {{ .Values.orchard.env | default "development" | quote }}
- name: ORCHARD_SERVER_HOST
value: {{ .Values.orchard.server.host | quote }}
- name: ORCHARD_SERVER_PORT
@@ -77,8 +79,16 @@ spec:
value: {{ include "orchard.postgresql.host" . | quote }}
- name: ORCHARD_DATABASE_PORT
value: {{ .Values.orchard.database.port | quote }}
{{- if and .Values.orchard.database.secretsManager .Values.orchard.database.secretsManager.enabled }}
- name: ORCHARD_DATABASE_USER
valueFrom:
secretKeyRef:
name: {{ include "orchard.postgresql.secretName" . }}
key: username
{{- else }}
- name: ORCHARD_DATABASE_USER
value: {{ .Values.orchard.database.user | default .Values.postgresql.auth.username | quote }}
{{- end }}
- name: ORCHARD_DATABASE_DBNAME
value: {{ .Values.orchard.database.dbname | default .Values.postgresql.auth.database | quote }}
- name: ORCHARD_DATABASE_SSLMODE
@@ -96,6 +106,7 @@ spec:
value: {{ .Values.orchard.s3.bucket | quote }}
- name: ORCHARD_S3_USE_PATH_STYLE
value: {{ .Values.orchard.s3.usePathStyle | quote }}
{{- if or .Values.minio.enabled .Values.orchard.s3.existingSecret .Values.orchard.s3.accessKeyId }}
- name: ORCHARD_S3_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
@@ -106,16 +117,38 @@ spec:
secretKeyRef:
name: {{ include "orchard.minio.secretName" . }}
key: {{ if .Values.minio.enabled }}root-password{{ else }}{{ .Values.orchard.s3.existingSecretSecretKeyKey }}{{ end }}
{{- end }}
- name: ORCHARD_DOWNLOAD_MODE
value: {{ .Values.orchard.download.mode | quote }}
- name: ORCHARD_PRESIGNED_URL_EXPIRY
value: {{ .Values.orchard.download.presignedUrlExpiry | quote }}
{{- if .Values.orchard.rateLimit }}
{{- if .Values.orchard.rateLimit.login }}
- name: ORCHARD_LOGIN_RATE_LIMIT
value: {{ .Values.orchard.rateLimit.login | quote }}
{{- end }}
{{- end }}
{{- if and .Values.orchard.database.secretsManager .Values.orchard.database.secretsManager.enabled }}
volumeMounts:
- name: db-secrets
mountPath: /mnt/secrets-store
readOnly: true
{{- end }}
livenessProbe:
{{- toYaml .Values.livenessProbe | nindent 12 }}
readinessProbe:
{{- toYaml .Values.readinessProbe | nindent 12 }}
resources:
{{- toYaml .Values.resources | nindent 12 }}
{{- if and .Values.orchard.database.secretsManager .Values.orchard.database.secretsManager.enabled }}
volumes:
- name: db-secrets
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: {{ include "orchard.fullname" . }}-db-secret
{{- end }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}

View File

@@ -0,0 +1,27 @@
{{- if and .Values.orchard.database.secretsManager .Values.orchard.database.secretsManager.enabled }}
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: {{ include "orchard.fullname" . }}-db-secret
labels:
{{- include "orchard.labels" . | nindent 4 }}
spec:
provider: aws
parameters:
objects: |
- objectName: "{{ .Values.orchard.database.secretsManager.secretArn }}"
objectType: "secretsmanager"
jmesPath:
- path: username
objectAlias: db-username
- path: password
objectAlias: db-password
secretObjects:
- secretName: {{ include "orchard.fullname" . }}-db-credentials
type: Opaque
data:
- objectName: db-username
key: username
- objectName: db-password
key: password
{{- end }}

View File

@@ -42,6 +42,7 @@ ingress:
className: "nginx"
annotations:
cert-manager.io/cluster-issuer: "letsencrypt"
nginx.ingress.kubernetes.io/proxy-body-size: "0" # Disable body size limit for uploads
hosts:
- host: orchard-dev.common.global.bsf.tools # Overridden by CI
paths:
@@ -84,6 +85,7 @@ tolerations: []
affinity: {}
orchard:
env: "development" # Allows seed data for testing
server:
host: "0.0.0.0"
port: 8080
@@ -113,6 +115,10 @@ orchard:
mode: "presigned"
presignedUrlExpiry: 3600
# Relaxed rate limits for dev/feature environments (allows integration tests to run)
rateLimit:
login: "1000/minute" # Default is 5/minute, relaxed for CI integration tests
# PostgreSQL - ephemeral, no persistence
postgresql:
enabled: true

View File

@@ -0,0 +1,136 @@
# Production values for orchard
# TODO: Replace subcharts with managed services (RDS, S3) when ready
replicaCount: 1
image:
repository: registry.global.bsf.tools/esv/bsf/bsf-integration/orchard/orchard-mvp
pullPolicy: Always
tag: "latest" # Overridden by CI
imagePullSecrets:
- name: orchard-pull-secret
initContainer:
image:
repository: containers.global.bsf.tools/busybox
tag: "1.36"
pullPolicy: IfNotPresent
serviceAccount:
create: true
automount: true
annotations:
eks.amazonaws.com/role-arn: arn:aws-us-gov:iam::052673043337:role/service-orchard-prd
name: "orchard"
podAnnotations: {}
podLabels: {}
podSecurityContext: {}
securityContext:
readOnlyRootFilesystem: false
runAsNonRoot: true
runAsUser: 1000
service:
type: ClusterIP
port: 8080
ingress:
enabled: true
className: "nginx"
annotations:
cert-manager.io/cluster-issuer: "letsencrypt"
nginx.ingress.kubernetes.io/proxy-body-size: "0" # Disable body size limit for uploads
hosts:
- host: orchard.common.global.bsf.tools
paths:
- path: /
pathType: Prefix
tls:
- secretName: orchard-prod-tls
hosts:
- orchard.common.global.bsf.tools
# Production resources - same as stage for MVP, increase as needed
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 500m
memory: 512Mi
livenessProbe:
httpGet:
path: /health
port: http
initialDelaySeconds: 10
periodSeconds: 10
readinessProbe:
httpGet:
path: /health
port: http
initialDelaySeconds: 5
periodSeconds: 5
autoscaling:
enabled: false
minReplicas: 1
maxReplicas: 10
targetCPUUtilizationPercentage: 80
targetMemoryUtilizationPercentage: 80
nodeSelector: {}
tolerations: []
affinity: {}
orchard:
env: "production" # Disables seed data
server:
host: "0.0.0.0"
port: 8080
# Database configuration - uses AWS Secrets Manager via CSI driver
database:
host: "orchard-prd.cluster-cvw3jzjkozoc.us-gov-west-1.rds.amazonaws.com"
port: 5432
dbname: orchard_prod
sslmode: require
secretsManager:
enabled: true
secretArn: "arn:aws-us-gov:secretsmanager:us-gov-west-1:052673043337:secret:rds!cluster-0afc8af5-f644-4284-92fb-2ed545490f92-3v9uXV"
# S3 configuration - uses IRSA for credentials
s3:
endpoint: "" # Empty = use AWS default
region: us-gov-west-1
bucket: orchard-artifacts-prod
usePathStyle: false # Real S3 uses virtual-hosted style
download:
mode: "presigned"
presignedUrlExpiry: 3600
# PostgreSQL subchart - disabled in prod, using RDS
postgresql:
enabled: false
# MinIO subchart - disabled in prod, using real S3
minio:
enabled: false
# MinIO ingress - disabled in prod, using real S3
minioIngress:
enabled: false
redis:
enabled: false
waitForDatabase: true
global:
security:
allowInsecureImages: true

View File

@@ -19,7 +19,8 @@ initContainer:
serviceAccount:
create: true
automount: true
annotations: {}
annotations:
eks.amazonaws.com/role-arn: arn:aws-us-gov:iam::052673043337:role/service-orchard-stage
name: "orchard"
podAnnotations: {}
@@ -41,6 +42,7 @@ ingress:
className: "nginx"
annotations:
cert-manager.io/cluster-issuer: "letsencrypt"
nginx.ingress.kubernetes.io/proxy-body-size: "0" # Disable body size limit for uploads
hosts:
- host: orchard-stage.common.global.bsf.tools
paths:
@@ -88,130 +90,48 @@ affinity: {}
# Orchard server configuration
orchard:
env: "development" # Allows seed data for testing
server:
host: "0.0.0.0"
port: 8080
# Database configuration (used when postgresql.enabled is false)
# Database configuration - uses AWS Secrets Manager via CSI driver
database:
host: ""
host: "orchard-stage.cluster-cvw3jzjkozoc.us-gov-west-1.rds.amazonaws.com"
port: 5432
user: orchard
password: ""
dbname: orchard
sslmode: disable
existingSecret: ""
existingSecretPasswordKey: "password"
dbname: postgres
sslmode: require
secretsManager:
enabled: true
secretArn: "arn:aws-us-gov:secretsmanager:us-gov-west-1:052673043337:secret:rds!cluster-a573672b-1a38-4665-a654-1b7df37b5297-IaeFQL"
# S3 configuration (used when minio.enabled is false)
# S3 configuration - uses IRSA for credentials
s3:
endpoint: ""
region: us-east-1
bucket: orchard-artifacts
accessKeyId: ""
secretAccessKey: ""
usePathStyle: true
existingSecret: ""
existingSecretAccessKeyKey: "access-key-id"
existingSecretSecretKeyKey: "secret-access-key"
endpoint: "" # Empty = use AWS default
region: us-gov-west-1
bucket: orchard-artifacts-stage
usePathStyle: false # Real S3 uses virtual-hosted style
# Download configuration
download:
mode: "presigned" # presigned, redirect, or proxy
presignedUrlExpiry: 3600 # Presigned URL expiry in seconds
# PostgreSQL subchart configuration
# Relaxed rate limits for stage (allows CI integration tests to run)
rateLimit:
login: "1000/minute" # Default is 5/minute, relaxed for CI integration tests
# PostgreSQL subchart - disabled in stage, using RDS
postgresql:
enabled: true
image:
registry: containers.global.bsf.tools
repository: bitnami/postgresql
tag: "15"
pullPolicy: IfNotPresent
auth:
username: orchard
password: orchard-password
database: orchard
primary:
persistence:
enabled: false
size: 10Gi
# Resources with memory requests = limits per cluster policy
resourcesPreset: "none"
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 250m
memory: 512Mi
# Volume permissions init container
volumePermissions:
resourcesPreset: "none"
resources:
limits:
cpu: 100m
memory: 128Mi
requests:
cpu: 50m
memory: 128Mi
enabled: false
# MinIO subchart configuration
# MinIO subchart - disabled in stage, using real S3
minio:
enabled: true
image:
registry: containers.global.bsf.tools
repository: bitnami/minio
tag: "latest"
pullPolicy: IfNotPresent
auth:
rootUser: minioadmin
rootPassword: minioadmin
defaultBuckets: "orchard-artifacts"
persistence:
enabled: false
size: 50Gi
# Resources with memory requests = limits per cluster policy
resourcesPreset: "none" # Disable preset to use explicit resources
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 250m
memory: 512Mi
# Init container resources
defaultInitContainers:
volumePermissions:
resourcesPreset: "none"
resources:
limits:
cpu: 100m
memory: 128Mi
requests:
cpu: 50m
memory: 128Mi
# Provisioning job resources
provisioning:
resources:
limits:
cpu: 200m
memory: 256Mi
requests:
cpu: 100m
memory: 256Mi
enabled: false
# MinIO external ingress for presigned URL access (separate from subchart ingress)
# MinIO ingress - disabled in stage, using real S3
minioIngress:
enabled: true
className: "nginx"
annotations:
cert-manager.io/cluster-issuer: "letsencrypt"
nginx.ingress.kubernetes.io/proxy-body-size: "0" # Disable body size limit for uploads
host: "minio-orch-stage.common.global.bsf.tools"
tls:
enabled: true
secretName: minio-tls
enabled: false
# Redis subchart configuration (for future caching)
redis:

View File

@@ -0,0 +1,67 @@
-- Migration: Add package_versions table for separate version tracking
-- This separates immutable versions from mutable tags
-- Create package_versions table
CREATE TABLE IF NOT EXISTS package_versions (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
package_id UUID NOT NULL REFERENCES packages(id) ON DELETE CASCADE,
artifact_id VARCHAR(64) NOT NULL REFERENCES artifacts(id),
version VARCHAR(255) NOT NULL,
version_source VARCHAR(50), -- 'explicit', 'filename', 'metadata', 'migrated_from_tag'
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
created_by VARCHAR(255) NOT NULL,
UNIQUE(package_id, version),
UNIQUE(package_id, artifact_id)
);
-- Indexes for common queries
CREATE INDEX IF NOT EXISTS idx_package_versions_package_id ON package_versions(package_id);
CREATE INDEX IF NOT EXISTS idx_package_versions_artifact_id ON package_versions(artifact_id);
CREATE INDEX IF NOT EXISTS idx_package_versions_package_version ON package_versions(package_id, version);
-- Trigger functions for ref_count management (same pattern as tags)
CREATE OR REPLACE FUNCTION increment_version_ref_count()
RETURNS TRIGGER AS $$
BEGIN
UPDATE artifacts SET ref_count = ref_count + 1 WHERE id = NEW.artifact_id;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION decrement_version_ref_count()
RETURNS TRIGGER AS $$
BEGIN
UPDATE artifacts SET ref_count = ref_count - 1 WHERE id = OLD.artifact_id;
RETURN OLD;
END;
$$ LANGUAGE plpgsql;
-- Create triggers
DROP TRIGGER IF EXISTS package_versions_ref_count_insert ON package_versions;
CREATE TRIGGER package_versions_ref_count_insert
AFTER INSERT ON package_versions
FOR EACH ROW
EXECUTE FUNCTION increment_version_ref_count();
DROP TRIGGER IF EXISTS package_versions_ref_count_delete ON package_versions;
CREATE TRIGGER package_versions_ref_count_delete
AFTER DELETE ON package_versions
FOR EACH ROW
EXECUTE FUNCTION decrement_version_ref_count();
-- Data migration: populate from existing semver-pattern tags
-- This extracts versions from tags that look like version numbers
-- Tags like "v1.0.0", "1.2.3", "2.0.0-beta" will be migrated
-- Tags like "latest", "stable", "dev" will NOT be migrated
INSERT INTO package_versions (package_id, artifact_id, version, version_source, created_by, created_at)
SELECT
t.package_id,
t.artifact_id,
-- Strip leading 'v' if present
CASE WHEN t.name LIKE 'v%' THEN substring(t.name from 2) ELSE t.name END,
'migrated_from_tag',
t.created_by,
t.created_at
FROM tags t
WHERE t.name ~ '^v?[0-9]+\.[0-9]+(\.[0-9]+)?([-.][a-zA-Z0-9]+)?$'
ON CONFLICT (package_id, version) DO NOTHING;

View File

@@ -0,0 +1,48 @@
-- Migration 008: Artifact Dependencies
-- Adds support for declaring dependencies between artifacts
-- Part of Package Dependency Management feature (#76)
-- Create artifact_dependencies table
CREATE TABLE IF NOT EXISTS artifact_dependencies (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
artifact_id VARCHAR(64) NOT NULL REFERENCES artifacts(id) ON DELETE CASCADE,
dependency_project VARCHAR(255) NOT NULL,
dependency_package VARCHAR(255) NOT NULL,
version_constraint VARCHAR(255),
tag_constraint VARCHAR(255),
created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
-- Exactly one of version_constraint or tag_constraint must be set
CONSTRAINT check_constraint_type CHECK (
(version_constraint IS NOT NULL AND tag_constraint IS NULL) OR
(version_constraint IS NULL AND tag_constraint IS NOT NULL)
),
-- Each artifact can only have one dependency on a specific project/package
CONSTRAINT unique_artifact_dependency UNIQUE (artifact_id, dependency_project, dependency_package)
);
-- Index for fast lookups by artifact_id (get all deps for an artifact)
CREATE INDEX IF NOT EXISTS idx_artifact_dependencies_artifact_id
ON artifact_dependencies(artifact_id);
-- Index for reverse dependency lookups (find what depends on a package)
CREATE INDEX IF NOT EXISTS idx_artifact_dependencies_target
ON artifact_dependencies(dependency_project, dependency_package);
-- Index for finding dependencies with specific version constraints
CREATE INDEX IF NOT EXISTS idx_artifact_dependencies_version
ON artifact_dependencies(dependency_project, dependency_package, version_constraint)
WHERE version_constraint IS NOT NULL;
-- Index for finding dependencies with specific tag constraints
CREATE INDEX IF NOT EXISTS idx_artifact_dependencies_tag
ON artifact_dependencies(dependency_project, dependency_package, tag_constraint)
WHERE tag_constraint IS NOT NULL;
COMMENT ON TABLE artifact_dependencies IS 'Stores dependencies declared by artifacts on other packages';
COMMENT ON COLUMN artifact_dependencies.artifact_id IS 'The artifact that declares this dependency';
COMMENT ON COLUMN artifact_dependencies.dependency_project IS 'Project name of the dependency';
COMMENT ON COLUMN artifact_dependencies.dependency_package IS 'Package name of the dependency';
COMMENT ON COLUMN artifact_dependencies.version_constraint IS 'Exact version required (mutually exclusive with tag_constraint)';
COMMENT ON COLUMN artifact_dependencies.tag_constraint IS 'Tag name required (mutually exclusive with version_constraint)';