bitforge/orchard
3
0
Fork 0
Code Projects Wiki Activity
78 Commits 19 Branches 0 Tags
e215ecabcd93f55834b0cea1324cba866d91d233
Commit Graph

78 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mondo Diaz
e215ecabcd Add S3 configuration options and improved error handling 
- Add s3_verify_ssl config option for SSL/TLS verification
- Add s3_connect_timeout and s3_read_timeout config options
- Add s3_max_retries config option with adaptive retry mode
- Add S3StorageUnavailableError for backend availability issues
- Add HashCollisionError for detecting extremely rare hash collisions
- Add hash collision detection by comparing file sizes on dedup
- Handle network interruption and timeout errors explicitly
- Update routes.py to handle new exception types with appropriate HTTP codes
 2026-01-05 14:46:18 -06:00
Mondo Diaz
eca291d194 Add S3 verification and failure cleanup tests 
- Add test_s3_bucket_single_object_after_duplicates to verify only one S3 object exists
- Add tests for upload failure scenarios (invalid project/package, empty file)
- Add tests for orphaned S3 objects and database records cleanup
- Add S3 direct access helpers (list_s3_objects_by_hash, s3_object_exists, etc.)
- Fix conftest.py to use setdefault for env vars (don't override container config)

All 52 tests now pass.
 2026-01-05 14:39:22 -06:00
Mondo Diaz
7c31b6a244 Add integration tests for deduplication and ref_count 
- Add test_integration_uploads.py with 12 tests for duplicate upload scenarios
- Add test_ref_count.py with 7 tests for ref_count management
- Fix ArtifactDetailResponse to include sha256 and checksum fields
- Fix health check SQL warning by wrapping in text()
- Update tests to use unique content per test run for idempotency
 2026-01-05 14:29:12 -06:00
Mondo Diaz
d2abfe671a Update CHANGELOG for issues #33, #34, #35, #36 2026-01-05 11:31:55 -06:00
Mondo Diaz
109677e43a Add storage abstraction, stats endpoints, garbage collection, and test infrastructure 
- Add StorageBackend protocol for backend-agnostic storage interface
- Add health check with storage and database connectivity verification
- Add garbage collection endpoints for orphaned artifacts (ref_count=0)
- Add deduplication statistics endpoints (/api/v1/stats, /stats/storage, /stats/deduplication)
- Add per-project statistics endpoint
- Add verify_integrity method for post-upload hash validation
- Set up pytest infrastructure with mock S3 client
- Add unit tests for hash calculation and duplicate detection
 2026-01-05 11:16:46 -06:00
Mondo Diaz
dbe78ded2f Fix double-counting: let SQL triggers manage ref_count 
Previously both Python code AND SQL triggers were incrementing/decrementing
ref_count, causing inconsistent values. Now:

- New artifacts start with ref_count=0 (triggers increment on tag creation)
- Tag INSERT/UPDATE/DELETE triggers handle all ref_count changes
- Python code only logs operations, doesn't manipulate ref_count
- Delete endpoints rely on cascade + triggers for ref_count management

Tested: new uploads, duplicates, tag deletion, package deletion (cascade),
project deletion (cascade) - all ref_count values now correct.
 2026-01-05 10:18:23 -06:00
Mondo Diaz
865812af98 Add ref_count management for deletions with atomic operations and error handling 
- Add DELETE endpoints for tags, packages, and projects with proper ref_count
  decrements for all affected artifacts
- Implement atomic ref_count operations using SELECT FOR UPDATE row-level locking
  to prevent race conditions
- Add custom storage exceptions (HashComputationError, S3ExistenceCheckError,
  S3UploadError) with retry logic for S3 existence checks
- Handle race conditions in upload by locking artifact row before modification
- Add comprehensive logging for all ref_count changes and deduplication events
- Include ref_count in upload response schema
 2026-01-05 10:04:59 -06:00
Mondo Diaz
66622caf5d Add AGENTS.md to gitignore for OpenCode compatibility 2026-01-05 09:24:33 -06:00
Mondo Diaz
96d79e4127 Merge branch 'fix/helm-minio-ingress-rename' into 'main' 
Fix Helm chart: rename minio.ingress to minioIngress to avoid subchart conflict

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!19
 2025-12-16 12:51:42 -06:00
Mondo Diaz
accba9e404 Fix Helm chart: rename minio.ingress to minioIngress to avoid subchart conflict 2025-12-16 12:51:41 -06:00
Dane Moss
64e420fb58 Merge branch 'cut_new_release' into 'main' 
release new image with presigned URL support for direct s3 downloads as default download mode

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!18
 2025-12-15 15:30:38 -07:00
dane.moss
994f166de8 release new image with presigned URL support for direct s3 downloads as default download mode 2025-12-15 15:17:53 -07:00
Mondo Diaz
8999552949 Merge branch 'feature/presigned-url-downloads' into 'main' 
Add presigned URL support for direct S3 downloads (#48)

Closes #48

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!17
 2025-12-15 16:06:51 -06:00
Mondo Diaz
2df97ae94a Add presigned URL support for direct S3 downloads (#48) 2025-12-15 16:06:51 -06:00
Mondo Diaz
caa0c5af0c Merge branch 'feature/store-sha256-checksums' into 'main' 
Store SHA256 checksums with artifacts and add multiple hash support

Closes #25

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!16
 2025-12-15 14:47:31 -06:00
Mondo Diaz
3fd2747ae4 Store SHA256 checksums with artifacts and add multiple hash support 2025-12-15 14:47:30 -06:00
Mondo Diaz
96367da448 Merge branch 'feature/integrity-verification-design' into 'main' 
Add integrity verification workflow design document

Closes #24

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!15
 2025-12-15 14:00:32 -06:00
Mondo Diaz
2686fdcb89 Add integrity verification workflow design document 2025-12-15 14:00:32 -06:00
Dane Moss
0eb2deb4ca Merge branch 'update_urls' into 'main' 
update URLs to point to BSF

Closes #46

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!14
 2025-12-15 11:30:07 -07:00
Dane Moss
3fe421f31d update URLs to point to BSF 2025-12-15 11:30:07 -07:00
Mondo Diaz
68660eacf6 Merge branch 'feature/schema-enhancements' into 'main' 
Add schema enhancements for uploads, artifacts, and audit tracking

Closes #16

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!13
 2025-12-12 15:23:50 -06:00
Mondo Diaz
b52c8840f1 Add schema enhancements for uploads, artifacts, and audit tracking 2025-12-12 15:23:50 -06:00
Dane Moss
4afcdf5cda Merge branch 'add_prosper_config' into 'main' 
Add prosper config

Closes #45

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!12
 2025-12-12 13:52:27 -07:00
Dane Moss
bc3da14d50 Add prosper config 2025-12-12 13:52:27 -07:00
Mondo Diaz
2843335f6d Document curl -OJ flag for correct download filenames 
- Update download examples to use -OJ flag
- Add note explaining curl download flags (-O, -J, -OJ, -o)
- Add example for saving to a specific filename
 2025-12-12 13:53:15 -06:00
Mondo Diaz
2097865874 Remove redundant search bar from Home, rename page filters 
- Remove SearchInput from Home page (use GlobalSearch in header instead)
- Rename "Search packages..." to "Filter packages..." on ProjectPage
- Rename "Search tags..." to "Filter tags..." on PackagePage
- Update FilterChip labels from "Search" to "Filter"

This differentiates the global search (header) from page-level filtering.
 2025-12-12 12:55:31 -06:00
Mondo Diaz
0e1474bf6c Merge branch 'feature/database-storage-layer' into 'main' 
Implement database storage layer

Closes #17

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!11
 2025-12-12 12:45:33 -06:00
Mondo Diaz
9604540dd3 Implement database storage layer 2025-12-12 12:45:33 -06:00
Mondo Diaz
a6df5aba5a Merge branch 'feature/search-filtering-enhancements' into 'main' 
Add global search and filtering enhancements

Closes #6

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!10
 2025-12-12 12:12:46 -06:00
Mondo Diaz
096887d4da Add global search and filtering enhancements 2025-12-12 12:12:46 -06:00
Mondo Diaz
7d80bef39a Fix: restore enhanced tags API endpoints 2025-12-12 10:57:27 -06:00
Mondo Diaz
96198dc127 Merge branch 'fix/restore-merged-features' 2025-12-12 10:55:19 -06:00
Mondo Diaz
fd06dfb3ce Reapply "Add API endpoints for listing tagged versions and artifacts" 
This reverts commit 11852adc66.
 2025-12-12 10:55:15 -06:00
Mondo Diaz
11852adc66 Revert "Add API endpoints for listing tagged versions and artifacts" 
This reverts commit 54e33e67ce.
 2025-12-12 10:49:55 -06:00
Mondo Diaz
21555d64a3 Merge branch 'fix/restore-merged-features' into 'main' 
fix merge issue

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!9
 2025-12-12 10:48:55 -06:00
Mondo Diaz
b83f19aa52 fix merge issue 2025-12-12 10:48:55 -06:00
Mondo Diaz
5d0122fc36 Revert "Add API endpoints for listing tagged versions and artifacts" 
This reverts commit 54e33e67ce.
 2025-12-12 10:33:21 -06:00
Mondo Diaz
81b423e0ea Merge branch 'feature/frontend-hierarchy-components' into 'main' 
Develop Frontend Components for Project, Package, and Instance Views

Closes #5

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!8
 2025-12-12 10:23:45 -06:00
Mondo Diaz
e89947f3d3 Develop Frontend Components for Project, Package, and Instance Views 2025-12-12 10:23:44 -06:00
Mondo Diaz
459867abdb Merge branch 'feature/list-tagged-versions-api' into 'main' 
Add API endpoints for listing tagged versions and artifacts

Closes #4

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!7
 2025-12-12 10:23:40 -06:00
Mondo Diaz
2b5bc60a69 Add API endpoints for listing tagged versions and artifacts 2025-12-12 10:23:40 -06:00
Mondo Diaz
8b7b523aa8 Merge branch 'feature/packages-api-enhancements' into 'main' 
Implement Backend API to List Packages within a Project

Closes #3

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!6
 2025-12-11 18:47:46 -06:00
Mondo Diaz
dea03c4a12 Implement Backend API to List Packages within a Project 2025-12-11 18:47:46 -06:00
Mondo Diaz
1793fd3a8f Merge branch 'feature/upload-download-apis' into 'main' 
Implement backend upload/download API enhancements

Closes #11

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!4
 2025-12-11 18:05:08 -06:00
Mondo Diaz
c119ab4a04 Implement backend upload/download API enhancements 2025-12-11 18:05:08 -06:00
Mondo Diaz
e9404a4425 Merge branch 'feature/projects-api-pagination-search' into 'main' 
Add pagination and search to projects API

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!1
 2025-12-11 15:03:42 -06:00
Mondo Diaz
b896ad1fad Add pagination and search to projects API 2025-12-11 15:03:41 -06:00
Mondo Diaz
b3861894cb Merge branch 'feature/modern-dark-ui' into 'main' 
Modern dark mode UI overhaul

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!3
 2025-12-11 14:58:13 -06:00
Mondo Diaz
71bb7d96b3 Modern dark mode UI overhaul 2025-12-11 14:58:13 -06:00
Mondo Diaz
e0802444c0 Merge branch 'feature/test-data-seeding' into 'main' 
Add development mode with automatic test data seeding

See merge request esv/bsf/bsf-integration/orchard/orchard-mvp!2
 2025-12-11 14:36:22 -06:00