feat: add auto-fetch for missing dependencies from upstream registries
Add auto_fetch parameter to dependency resolution endpoint that fetches missing dependencies from upstream registries (PyPI) when resolving. - Add RegistryClient abstraction with PyPIRegistryClient implementation - Extract fetch_and_cache_pypi_package() for reuse - Add resolve_dependencies_with_fetch() async function - Extend MissingDependency schema with fetch_attempted/fetch_error - Add fetched list to DependencyResolutionResponse - Add auto_fetch_max_depth config setting (default: 3) - Remove Usage section from Package page UI - Add 6 integration tests for auto-fetch functionality
This commit is contained in:
12
CHANGELOG.md
12
CHANGELOG.md
@@ -7,6 +7,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
### Added
|
||||
- Added auto-fetch capability to dependency resolution endpoint
|
||||
- `GET /api/v1/project/{project}/{package}/+/{ref}/resolve?auto_fetch=true` fetches missing dependencies from upstream registries
|
||||
- PyPI registry client queries PyPI JSON API to resolve version constraints
|
||||
- Fetched artifacts are cached and included in response `fetched` field
|
||||
- Missing dependencies show `fetch_attempted` and `fetch_error` status
|
||||
- Configurable max fetch depth via `ORCHARD_AUTO_FETCH_MAX_DEPTH` (default: 3)
|
||||
- Added `backend/app/registry_client.py` with extensible registry client abstraction
|
||||
- `RegistryClient` ABC for implementing upstream registry clients
|
||||
- `PyPIRegistryClient` implementation using PyPI JSON API
|
||||
- `get_registry_client()` factory function for future npm/maven support
|
||||
- Added `fetch_and_cache_pypi_package()` reusable function for PyPI package fetching
|
||||
- Added HTTP connection pooling infrastructure for improved PyPI proxy performance
|
||||
- `HttpClientManager` with configurable pool size, timeouts, and thread pool executor
|
||||
- Eliminates per-request connection overhead (~100-500ms → ~5ms)
|
||||
@@ -33,6 +44,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
- Added `POST /api/v1/cache/resolve` endpoint to cache packages by coordinates instead of URL (#108)
|
||||
|
||||
### Changed
|
||||
- Removed Usage section from Package page (curl command examples)
|
||||
- PyPI proxy now uses shared HTTP connection pool instead of per-request clients
|
||||
- PyPI proxy now caches upstream source configuration in Redis
|
||||
- Dependency storage now uses batch INSERT instead of individual queries
|
||||
|
||||
Reference in New Issue
Block a user