Commit Graph

24 Commits

Author SHA1 Message Date
Mondo Diaz
262aff6e97 fix: add security checks and tests for code review
Security:
- Add authorization checks to list_packages, update_package, delete_package endpoints
- Add MAX_TOTAL_ARTIFACTS limit (1000) to prevent memory exhaustion during dependency resolution
- Add TooManyArtifactsError exception for proper error handling

UI:
- Display reverse dependency errors in PackagePage
- Add warning display for failed dependency fetches in DependencyGraph

Tests:
- Add unit tests for metadata extraction (deb, wheel, tarball, jar)
- Add unit tests for rate limit configuration
- Add unit tests for PyPI registry client
2026-02-05 09:15:48 -06:00
Mondo Diaz
1389a03c69 fix: filter platform-specific and extra dependencies in PyPI proxy
The dependency parser was stripping environment markers but not checking
if they indicated optional or platform-specific packages. This caused
packages like jaraco.path to pull in pyobjc (324 sub-packages) even on
non-macOS systems.

Changes:
- Filter dependencies with 'extra ==' markers (optional extras)
- Filter dependencies with 'sys_platform' or 'platform_system' markers
- Add diagnostic logging for depth exceeded errors
- Add unit tests for dependency filtering

Fixes tensorflow dependency resolution exceeding max depth.
2026-02-05 09:15:48 -06:00
Mondo Diaz
a45ec46e94 chore: increase MAX_DEPENDENCY_DEPTH from 50 to 100 2026-02-05 09:15:48 -06:00
Mondo Diaz
f5c9e438a0 feat: remove fetch depth limit for dependency resolution
Real package managers (pip, npm, Maven) don't have depth limits - they
resolve the full dependency tree. We have other safeguards:
- Loop prevention via fetch_attempted set
- Timeout via auto_fetch_timeout setting
- Dependency trees are finite
2026-02-05 09:15:48 -06:00
Mondo Diaz
aff08ad393 fix: use lenient conflict handling for dependency resolution
Instead of failing with 409 on version conflicts, use "first version wins"
strategy. This allows resolution to succeed for complex dependency trees
like tensorflow where transitive dependencies may have overlapping but
not identical version requirements.

The resolver now:
- Checks if an already-resolved version satisfies a new constraint
- If yes, reuses the existing version
- If no, logs the mismatch and uses the first-encountered version

This matches pip's behavior of picking a working version rather than
failing on theoretical conflicts.
2026-02-05 09:15:48 -06:00
Mondo Diaz
cdb3b5ecb3 feat: increase auto_fetch_max_depth from 3 to 10 2026-02-05 09:15:48 -06:00
Mondo Diaz
659ecf6f73 fix: prevent false circular dependency detection on self-dependencies
When packages like pytest have extras (e.g., pytest[testing]) that depend
on the base package, the resolution was incorrectly detecting this as a
circular dependency.

Added additional check to skip dependencies that resolve to an artifact
already in the visiting set, preventing the false cycle detection while
still catching real circular dependencies.
2026-02-05 09:15:48 -06:00
Mondo Diaz
65bb073a6e fix: fetch root artifact from upstream when missing in auto_fetch mode
When auto_fetch=true and the root artifact doesn't exist locally in a
system project (_pypi), now attempts to fetch it from upstream before
starting dependency resolution. Also fixed a bug where fetched_artifacts
was being redeclared, which would lose the root artifact from the list.
2026-02-05 09:15:48 -06:00
Mondo Diaz
cbc2e5e11a feat: add auto-fetch for missing dependencies from upstream registries
Add auto_fetch parameter to dependency resolution endpoint that fetches
missing dependencies from upstream registries (PyPI) when resolving.

- Add RegistryClient abstraction with PyPIRegistryClient implementation
- Extract fetch_and_cache_pypi_package() for reuse
- Add resolve_dependencies_with_fetch() async function
- Extend MissingDependency schema with fetch_attempted/fetch_error
- Add fetched list to DependencyResolutionResponse
- Add auto_fetch_max_depth config setting (default: 3)
- Remove Usage section from Package page UI
- Add 6 integration tests for auto-fetch functionality
2026-02-05 09:15:48 -06:00
Mondo Diaz
308057784e Fix tests for tag removal and version behavior
- Fix upload response to return actual version (not requested version)
  when artifact already has a version in the package
- Update ref_count tests to use multiple packages (one version per
  artifact per package design constraint)
- Remove allow_public_internet references from upstream caching tests
- Update consistency check test to not assert global system health
- Add versions field to artifact schemas
- Fix dependencies resolution to handle removed tag constraint
2026-02-05 09:15:09 -06:00
Mondo Diaz
86e971381a Remove tag system, use versions only for artifact references
Tags were mutable aliases that caused confusion alongside the immutable
version system. This removes tags entirely, keeping only PackageVersion
for artifact references.

Changes:
- Remove tags and tag_history tables (migration 012)
- Remove Tag model, TagRepository, and 6 tag API endpoints
- Update cache system to create versions instead of tags
- Update frontend to display versions instead of tags
- Remove tag-related schemas and types
- Update artifact cleanup service for version-based ref_count
2026-02-05 09:15:09 -06:00
Mondo Diaz
9313942f53 Fix self-dependency detection to strip PyPI extras brackets
The circular dependency error '_pypi/psutil → _pypi/psutil' occurred because
dependencies with extras like 'psutil[test]' weren't being recognized as
self-dependencies. The comparison 'psutil[test] != psutil' failed.

- Add _normalize_pypi_package_name() helper that strips extras brackets
  and normalizes separators per PEP 503
- Update _detect_package_cycle to use normalized names for cycle detection
- Update check_circular_dependencies to use normalized initial path
- Simplify self-dependency check in resolve_dependencies to use helper
2026-02-05 09:15:09 -06:00
Mondo Diaz
9a795a301a Fix circular dependency resolution by switching to artifact-centric display
- Add artifact: prefix handling in resolve_dependencies for direct artifact
  ID references, enabling dependency resolution for tagless artifacts
- Refactor PackagePage from tag-based to artifact-based data display
- Add PackageArtifact type with tags array for artifact-centric API responses
- Update download URLs to use artifact:ID prefix when no tags exist
- Conditionally show "View Ensure File" only when artifact has tags
2026-02-05 09:15:09 -06:00
Mondo Diaz
d422ed5cd8 Fix self-dependency check to use case-insensitive PyPI name normalization 2026-02-05 09:15:09 -06:00
Mondo Diaz
34d98f52cb Fix circular dependency error message to show actual cycle path
The error was hardcoding [pkg_key, pkg_key] regardless of actual cycle.
Now tracks the path through dependencies to report the real cycle.
2026-02-05 09:15:09 -06:00
Mondo Diaz
0b85f37abd Fix circular dependency detection and hide empty graph modal
- Add artifact-level self-dependency check (skip if dep resolves to same artifact)
- Close dependency graph modal if package has no dependencies to show
  (only root package with no children and no missing deps)
2026-02-05 09:15:09 -06:00
Mondo Diaz
101152f87f Skip self-dependencies in dependency resolver
PyPI packages can have self-referential dependencies for extras
(e.g., pytest[testing] depends on pytest). These were incorrectly
detected as circular dependencies. Now we skip them.
2026-02-05 09:15:09 -06:00
Mondo Diaz
88765b4f50 Show missing dependencies in dependency graph instead of failing
When dependencies are not cached on the server (common since we removed
proactive caching), the dependency graph now:
- Continues resolving what it can find
- Shows missing dependencies in a separate section with amber styling
- Displays the constraint and which package required them
- Updates the header stats to show "X cached • Y not cached"

This provides a better user experience than showing an error when
some dependencies haven't been downloaded yet.
2026-02-05 09:15:09 -06:00
Mondo Diaz
152af0a852 Fix dependency graph error for invalid version constraints
When a dependency has an invalid version constraint like '>=' (without
a version number), the resolver now treats it as a wildcard and returns
the latest available version instead of failing with 'Dependency not found'.

This handles malformed metadata that may have been stored from PyPI packages.
2026-02-05 09:15:09 -06:00
Mondo Diaz
e62e75bade Fix duplicate dependency constraint causing 500 errors
- Deduplicate dependencies by package name before inserting
- Some packages (like anyio) list the same dep (trio) multiple times with
  different version constraints for different extras
- The unique constraint on (artifact_id, project, package) rejected these
- Also removed debug logging from dependencies.py
2026-02-05 09:15:08 -06:00
Mondo Diaz
befa517485 Add detailed debug logging to _resolve_dependency_to_artifact 2026-02-05 09:15:08 -06:00
Mondo Diaz
7a2c0a54c6 Add debug logging to resolve_dependencies 2026-02-05 09:15:08 -06:00
Mondo Diaz
4b76ca2046 Add PEP 440 version constraint matching for dependency resolution
- Parse version constraints like >=1.9, <2.0 using packaging library
- Find the latest version that satisfies the constraint
- Support wildcard (*) to get latest version
- Fall back to exact version and tag matching
2026-02-05 09:15:08 -06:00
Mondo Diaz
abba90ebac Add package dependencies system and project settings page 2026-01-27 10:11:04 -06:00