Compare commits
10 Commits
pipeline
...
50bcd35e68
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
50bcd35e68 | ||
|
|
61cd5c471a | ||
|
|
e83ecf4717 | ||
|
|
53b37c2e16 | ||
|
|
a9bf480954 | ||
|
|
4afbc53420 | ||
|
|
d0594fb161 | ||
|
|
7a0e0c95aa | ||
| 943cd6935b | |||
| 4a270dbfe3 |
@@ -1,20 +1,19 @@
|
|||||||
stages:
|
stages:
|
||||||
- build
|
|
||||||
- deploy
|
- deploy
|
||||||
|
|
||||||
|
|
||||||
build_container:
|
# build_container:
|
||||||
stage: build
|
# stage: build
|
||||||
image: deps.global.bsf.tools/quay.io/buildah/stable:latest
|
# image: deps.global.bsf.tools/quay.io/buildah/stable:latest
|
||||||
variables:
|
# variables:
|
||||||
IMAGE_NAME: "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME"
|
# IMAGE_NAME: "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME"
|
||||||
before_script:
|
# before_script:
|
||||||
- mkdir -p /tmp/buildah-storage
|
# - mkdir -p /tmp/buildah-storage
|
||||||
- export BUILDAH_ROOT="/tmp/buildah-storage"
|
# - export BUILDAH_ROOT="/tmp/buildah-storage"
|
||||||
- echo "$CI_REGISTRY_PASSWORD" | buildah login -u "$CI_REGISTRY_USER" --password-stdin "$CI_REGISTRY"
|
# - echo "$CI_REGISTRY_PASSWORD" | buildah login -u "$CI_REGISTRY_USER" --password-stdin "$CI_REGISTRY"
|
||||||
script:
|
# script:
|
||||||
- buildah bud --build-arg NPM_REGISTRY=https://deps.global.bsf.tools/artifactory/api/npm/registry.npmjs.org/ --storage-driver vfs --isolation chroot -t $IMAGE_NAME .
|
# - buildah bud --build-arg NPM_REGISTRY=https://deps.global.bsf.tools/artifactory/api/npm/registry.npmjs.org/ --storage-driver vfs --isolation chroot -t $IMAGE_NAME .
|
||||||
- buildah push --storage-driver vfs $IMAGE_NAME
|
# - buildah push --storage-driver vfs $IMAGE_NAME
|
||||||
|
|
||||||
deploy_helm_charts:
|
deploy_helm_charts:
|
||||||
stage: deploy
|
stage: deploy
|
||||||
@@ -35,12 +34,4 @@ deploy_helm_charts:
|
|||||||
script:
|
script:
|
||||||
- kubectl config use-context $CONTEXT
|
- kubectl config use-context $CONTEXT
|
||||||
- |
|
- |
|
||||||
helm upgrade --install warehouse13-$CI_COMMIT_REF_NAME \
|
helm upgrade --install warehouse13-$CI_COMMIT_REF_NAME ./helm/warehouse13 --namespace $NAMESPACE -f $VALUES_FILE
|
||||||
./helm/warehouse13 --namespace $NAMESPACE \
|
|
||||||
-f $VALUES_FILE \
|
|
||||||
--set api.image=$CI_REGISTRY_IMAGE \
|
|
||||||
--set api.image.tag=$CI_COMMIT_REF_NAME \
|
|
||||||
--set postgres.image.repository=containers.global.bsf.tools/postgres \
|
|
||||||
--set postgres.image.tag=15-alpine \
|
|
||||||
--set minio.image.repository=containers.global.bsf.tools/minio \
|
|
||||||
--set minio.image.tag=latest
|
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
# Multi-stage build: First stage builds Angular frontend
|
# Multi-stage build: First stage builds Angular frontend
|
||||||
FROM node:24-alpine AS frontend-build
|
FROM node:20.11-alpine3.19 AS frontend-build
|
||||||
|
|
||||||
# Accept npm registry as build argument
|
# Accept npm registry as build argument
|
||||||
ARG NPM_REGISTRY=https://registry.npmjs.org/
|
ARG NPM_REGISTRY=https://registry.npmjs.org/
|
||||||
|
|||||||
220
docs/NPM-PACKAGE-AGE-POLICY.md
Normal file
220
docs/NPM-PACKAGE-AGE-POLICY.md
Normal file
@@ -0,0 +1,220 @@
|
|||||||
|
# NPM Package Age Policy
|
||||||
|
|
||||||
|
## Requirement
|
||||||
|
|
||||||
|
All npm packages must be **at least 2 weeks old** before they can be used in this project. This ensures:
|
||||||
|
- Package stability
|
||||||
|
- Security vulnerability disclosure time
|
||||||
|
- Compliance with organizational security policies
|
||||||
|
|
||||||
|
## How It Works
|
||||||
|
|
||||||
|
### 1. Package Version Pinning
|
||||||
|
|
||||||
|
The project uses exact version pinning in `package.json` to prevent automatic updates:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"dependencies": {
|
||||||
|
"@angular/core": "19.2.7", // Exact version, not "^19.2.7" or "~19.2.7"
|
||||||
|
// ...
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The `frontend/.npmrc` file enforces this:
|
||||||
|
```
|
||||||
|
save-exact=true
|
||||||
|
package-lock=true
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Automated Age Checking
|
||||||
|
|
||||||
|
Use the provided script to verify all packages meet the 2-week requirement:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check if all packages are at least 2 weeks old
|
||||||
|
node scripts/check-package-age.js
|
||||||
|
```
|
||||||
|
|
||||||
|
This script:
|
||||||
|
- Queries npm registry for publish dates
|
||||||
|
- Calculates age of each package
|
||||||
|
- Fails if any package is newer than 14 days
|
||||||
|
- Shows detailed age information for all packages
|
||||||
|
|
||||||
|
### 3. Installation Process
|
||||||
|
|
||||||
|
**Always use `npm ci` instead of `npm install`:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd frontend
|
||||||
|
npm ci # Installs exact versions from package-lock.json
|
||||||
|
```
|
||||||
|
|
||||||
|
**Why `npm ci`?**
|
||||||
|
- Uses exact versions from `package-lock.json`
|
||||||
|
- Doesn't update `package-lock.json`
|
||||||
|
- Ensures reproducible builds
|
||||||
|
- Faster than `npm install`
|
||||||
|
|
||||||
|
## Updating Packages
|
||||||
|
|
||||||
|
When you need to add or update packages:
|
||||||
|
|
||||||
|
### Step 1: Add Package to package.json
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Find a version that's at least 2 weeks old
|
||||||
|
npm view <package-name> time
|
||||||
|
|
||||||
|
# Add exact version to package.json
|
||||||
|
"dependencies": {
|
||||||
|
"new-package": "1.2.3"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 2: Verify Age
|
||||||
|
|
||||||
|
```bash
|
||||||
|
node scripts/check-package-age.js
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 3: Update Lock File
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd frontend
|
||||||
|
npm install --package-lock-only
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 4: Install and Test
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm ci
|
||||||
|
npm run build:prod
|
||||||
|
```
|
||||||
|
|
||||||
|
## CI/CD Integration
|
||||||
|
|
||||||
|
Add the age check to your CI/CD pipeline:
|
||||||
|
|
||||||
|
### GitLab CI Example
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
verify_package_age:
|
||||||
|
stage: validate
|
||||||
|
image: node:18-alpine
|
||||||
|
script:
|
||||||
|
- node scripts/check-package-age.js
|
||||||
|
only:
|
||||||
|
- merge_requests
|
||||||
|
- main
|
||||||
|
```
|
||||||
|
|
||||||
|
### GitHub Actions Example
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
- name: Check Package Age
|
||||||
|
run: node scripts/check-package-age.js
|
||||||
|
```
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### "Package is too new" Error
|
||||||
|
|
||||||
|
If a package fails the age check:
|
||||||
|
|
||||||
|
1. **Find an older version:**
|
||||||
|
```bash
|
||||||
|
npm view <package-name> versions --json
|
||||||
|
npm view <package-name>@<older-version> time
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Update package.json with older version**
|
||||||
|
|
||||||
|
3. **Re-run age check:**
|
||||||
|
```bash
|
||||||
|
node scripts/check-package-age.js
|
||||||
|
```
|
||||||
|
|
||||||
|
### Can't Find Old Enough Version
|
||||||
|
|
||||||
|
If no version meets the 2-week requirement:
|
||||||
|
- Wait until the package is at least 2 weeks old
|
||||||
|
- Look for alternative packages
|
||||||
|
- Request an exception through your security team
|
||||||
|
|
||||||
|
## Example Workflow
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Check current package ages
|
||||||
|
node scripts/check-package-age.js
|
||||||
|
|
||||||
|
# 2. If all pass, install dependencies
|
||||||
|
cd frontend
|
||||||
|
npm ci
|
||||||
|
|
||||||
|
# 3. Build application
|
||||||
|
npm run build:prod
|
||||||
|
|
||||||
|
# 4. For air-gapped deployment
|
||||||
|
../scripts/build-for-airgap.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
## Scripts Reference
|
||||||
|
|
||||||
|
| Script | Purpose |
|
||||||
|
|--------|---------|
|
||||||
|
| `scripts/check-package-age.js` | Verify all packages are ≥ 2 weeks old |
|
||||||
|
| `scripts/pin-old-versions.sh` | Helper script to validate and pin versions |
|
||||||
|
| `scripts/build-for-airgap.sh` | Build frontend for air-gapped deployment |
|
||||||
|
|
||||||
|
## Best Practices
|
||||||
|
|
||||||
|
1. **Always commit `package-lock.json`**
|
||||||
|
- Ensures everyone uses the same versions
|
||||||
|
- Required for reproducible builds
|
||||||
|
|
||||||
|
2. **Use `npm ci` in CI/CD**
|
||||||
|
- Faster than `npm install`
|
||||||
|
- Enforces lock file versions
|
||||||
|
- Prevents surprises
|
||||||
|
|
||||||
|
3. **Regular audits**
|
||||||
|
```bash
|
||||||
|
# Check for security vulnerabilities
|
||||||
|
npm audit
|
||||||
|
|
||||||
|
# Check package ages
|
||||||
|
node scripts/check-package-age.js
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Version ranges to avoid**
|
||||||
|
- ❌ `^1.2.3` (allows minor/patch updates)
|
||||||
|
- ❌ `~1.2.3` (allows patch updates)
|
||||||
|
- ❌ `*` or `latest` (allows any version)
|
||||||
|
- ✅ `1.2.3` (exact version only)
|
||||||
|
|
||||||
|
## Package Age Check Output
|
||||||
|
|
||||||
|
```
|
||||||
|
Checking package ages (must be at least 2 weeks old)...
|
||||||
|
|
||||||
|
✓ @angular/common@19.2.7 - 45 days old
|
||||||
|
✓ @angular/compiler@19.2.7 - 45 days old
|
||||||
|
✓ rxjs@7.8.0 - 180 days old
|
||||||
|
❌ new-package@1.0.0 - 5 days old (published 2025-01-12)
|
||||||
|
|
||||||
|
================================================================================
|
||||||
|
|
||||||
|
❌ FAILED: 1 package(s) are newer than 2 weeks:
|
||||||
|
|
||||||
|
- new-package@1.0.0 (5 days old, published 2025-01-12)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Support
|
||||||
|
|
||||||
|
For questions or exceptions:
|
||||||
|
- Review with security team
|
||||||
|
- Document in project README
|
||||||
|
- Update this policy as needed
|
||||||
@@ -18,6 +18,8 @@ spec:
|
|||||||
{{- include "warehouse13.selectorLabels" . | nindent 8 }}
|
{{- include "warehouse13.selectorLabels" . | nindent 8 }}
|
||||||
app.kubernetes.io/component: app
|
app.kubernetes.io/component: app
|
||||||
spec:
|
spec:
|
||||||
|
imagePullSecrets:
|
||||||
|
- name: gitlab-service-agent-registry-secret
|
||||||
serviceAccountName: {{ include "warehouse13.serviceAccountName" . }}
|
serviceAccountName: {{ include "warehouse13.serviceAccountName" . }}
|
||||||
securityContext:
|
securityContext:
|
||||||
{{- toYaml .Values.podSecurityContext | nindent 8 }}
|
{{- toYaml .Values.podSecurityContext | nindent 8 }}
|
||||||
|
|||||||
@@ -10,9 +10,9 @@ global:
|
|||||||
postgres:
|
postgres:
|
||||||
enabled: true
|
enabled: true
|
||||||
image:
|
image:
|
||||||
repository: postgres
|
repository: containers.global.bsf.tools/postgres
|
||||||
tag: 15-alpine
|
tag: 15-alpine
|
||||||
pullPolicy: always
|
pullPolicy: Always
|
||||||
auth:
|
auth:
|
||||||
username: user
|
username: user
|
||||||
password: password
|
password: password
|
||||||
@@ -36,9 +36,9 @@ postgres:
|
|||||||
minio:
|
minio:
|
||||||
enabled: true
|
enabled: true
|
||||||
image:
|
image:
|
||||||
repository: minio/minio
|
repository: containers.global.bsf.tools/minio/minio
|
||||||
tag: latest
|
tag: latest
|
||||||
pullPolicy: always
|
pullPolicy: Always
|
||||||
auth:
|
auth:
|
||||||
rootUser: minioadmin
|
rootUser: minioadmin
|
||||||
rootPassword: minioadmin
|
rootPassword: minioadmin
|
||||||
@@ -65,9 +65,9 @@ minio:
|
|||||||
app:
|
app:
|
||||||
enabled: true
|
enabled: true
|
||||||
image:
|
image:
|
||||||
repository: warehouse13/app
|
repository: registry.global.bsf.tools/esv/bsf/bsf-services/warehouse13
|
||||||
tag: latest
|
tag: main-7126c618
|
||||||
pullPolicy: always
|
pullPolicy: Always
|
||||||
replicas: 2
|
replicas: 2
|
||||||
env:
|
env:
|
||||||
databaseUrl: "postgresql://user:password@warehouse13-postgres:5432/warehouse13"
|
databaseUrl: "postgresql://user:password@warehouse13-postgres:5432/warehouse13"
|
||||||
|
|||||||
146
scripts/check-package-age.js
Executable file
146
scripts/check-package-age.js
Executable file
@@ -0,0 +1,146 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Check if npm packages are at least 2 weeks old before installation
|
||||||
|
* Usage: node scripts/check-package-age.js
|
||||||
|
*/
|
||||||
|
|
||||||
|
const https = require('https');
|
||||||
|
const fs = require('fs');
|
||||||
|
const path = require('path');
|
||||||
|
|
||||||
|
const TWO_WEEKS_MS = 14 * 24 * 60 * 60 * 1000;
|
||||||
|
|
||||||
|
function getPackageInfo(packageName, version) {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
// Remove version prefixes like ^, ~, >=
|
||||||
|
let cleanVersion = version.replace(/^[\^~>=]+/, '');
|
||||||
|
|
||||||
|
// If version contains .x or wildcards, fetch all versions and find latest matching
|
||||||
|
if (cleanVersion.includes('x') || cleanVersion.includes('*')) {
|
||||||
|
const url = `https://registry.npmjs.org/${packageName}`;
|
||||||
|
|
||||||
|
https.get(url, (res) => {
|
||||||
|
let data = '';
|
||||||
|
|
||||||
|
res.on('data', (chunk) => {
|
||||||
|
data += chunk;
|
||||||
|
});
|
||||||
|
|
||||||
|
res.on('end', () => {
|
||||||
|
try {
|
||||||
|
const info = JSON.parse(data);
|
||||||
|
const versions = Object.keys(info.versions || {});
|
||||||
|
|
||||||
|
// Convert version pattern to regex (e.g., "19.2.x" -> /^19\.2\.\d+$/)
|
||||||
|
const pattern = cleanVersion.replace(/\./g, '\\.').replace(/x|\*/g, '\\d+');
|
||||||
|
const regex = new RegExp(`^${pattern}$`);
|
||||||
|
|
||||||
|
// Find matching versions and get the latest
|
||||||
|
const matchingVersions = versions.filter(v => regex.test(v)).sort();
|
||||||
|
const latestMatching = matchingVersions[matchingVersions.length - 1];
|
||||||
|
|
||||||
|
if (latestMatching && info.time && info.time[latestMatching]) {
|
||||||
|
resolve({
|
||||||
|
name: packageName,
|
||||||
|
version: latestMatching,
|
||||||
|
published: info.time[latestMatching],
|
||||||
|
error: null
|
||||||
|
});
|
||||||
|
} else {
|
||||||
|
reject(new Error(`No matching version found for ${packageName}@${cleanVersion}`));
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
reject(new Error(`Failed to parse response for ${packageName}@${cleanVersion}`));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}).on('error', (err) => {
|
||||||
|
reject(err);
|
||||||
|
});
|
||||||
|
} else {
|
||||||
|
// Exact version lookup
|
||||||
|
const url = `https://registry.npmjs.org/${packageName}/${cleanVersion}`;
|
||||||
|
|
||||||
|
https.get(url, (res) => {
|
||||||
|
let data = '';
|
||||||
|
|
||||||
|
res.on('data', (chunk) => {
|
||||||
|
data += chunk;
|
||||||
|
});
|
||||||
|
|
||||||
|
res.on('end', () => {
|
||||||
|
try {
|
||||||
|
const info = JSON.parse(data);
|
||||||
|
resolve({
|
||||||
|
name: packageName,
|
||||||
|
version: cleanVersion,
|
||||||
|
published: info.time || info._time,
|
||||||
|
error: null
|
||||||
|
});
|
||||||
|
} catch (err) {
|
||||||
|
reject(new Error(`Failed to parse response for ${packageName}@${cleanVersion}`));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}).on('error', (err) => {
|
||||||
|
reject(err);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
async function checkPackageAge() {
|
||||||
|
const packageJsonPath = path.join(__dirname, '../frontend/package.json');
|
||||||
|
const packageJson = JSON.parse(fs.readFileSync(packageJsonPath, 'utf8'));
|
||||||
|
|
||||||
|
const allDeps = {
|
||||||
|
...packageJson.dependencies,
|
||||||
|
...packageJson.devDependencies
|
||||||
|
};
|
||||||
|
|
||||||
|
console.log('Checking package ages (must be at least 2 weeks old)...\n');
|
||||||
|
|
||||||
|
const tooNew = [];
|
||||||
|
const errors = [];
|
||||||
|
|
||||||
|
for (const [name, version] of Object.entries(allDeps)) {
|
||||||
|
try {
|
||||||
|
const info = await getPackageInfo(name, version);
|
||||||
|
const publishDate = new Date(info.published);
|
||||||
|
const age = Date.now() - publishDate.getTime();
|
||||||
|
const ageInDays = Math.floor(age / (24 * 60 * 60 * 1000));
|
||||||
|
|
||||||
|
if (age < TWO_WEEKS_MS) {
|
||||||
|
tooNew.push({
|
||||||
|
name,
|
||||||
|
version: info.version,
|
||||||
|
age: ageInDays,
|
||||||
|
published: publishDate.toISOString().split('T')[0]
|
||||||
|
});
|
||||||
|
console.log(`❌ ${name}@${info.version} - ${ageInDays} days old (published ${publishDate.toISOString().split('T')[0]})`);
|
||||||
|
} else {
|
||||||
|
console.log(`✓ ${name}@${info.version} - ${ageInDays} days old`);
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
errors.push({ name, version, error: err.message });
|
||||||
|
console.log(`⚠️ ${name}@${version} - Could not check: ${err.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log('\n' + '='.repeat(80));
|
||||||
|
|
||||||
|
if (tooNew.length > 0) {
|
||||||
|
console.log(`\n❌ FAILED: ${tooNew.length} package(s) are newer than 2 weeks:\n`);
|
||||||
|
tooNew.forEach(pkg => {
|
||||||
|
console.log(` - ${pkg.name}@${pkg.version} (${pkg.age} days old, published ${pkg.published})`);
|
||||||
|
});
|
||||||
|
process.exit(1);
|
||||||
|
} else if (errors.length > 0) {
|
||||||
|
console.log(`\n⚠️ WARNING: Could not verify ${errors.length} package(s)`);
|
||||||
|
process.exit(0);
|
||||||
|
} else {
|
||||||
|
console.log('\n✓ SUCCESS: All packages are at least 2 weeks old');
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
checkPackageAge();
|
||||||
41
scripts/pin-old-versions.sh
Executable file
41
scripts/pin-old-versions.sh
Executable file
@@ -0,0 +1,41 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
# Pin npm packages to versions that are at least 2 weeks old
|
||||||
|
# This script helps ensure compliance with package age requirements
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
echo "========================================="
|
||||||
|
echo "Pin NPM Packages to Old Versions"
|
||||||
|
echo "========================================="
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
cd frontend
|
||||||
|
|
||||||
|
echo "Step 1: Checking current package ages..."
|
||||||
|
node ../scripts/check-package-age.js || {
|
||||||
|
echo ""
|
||||||
|
echo "Some packages are too new. Recommendations:"
|
||||||
|
echo "1. Manually downgrade packages in package.json to older versions"
|
||||||
|
echo "2. Run: npm install --package-lock-only to update lock file"
|
||||||
|
echo "3. Re-run this script to verify"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "Step 2: Ensuring package-lock.json uses exact versions..."
|
||||||
|
if [ -f "package-lock.json" ]; then
|
||||||
|
echo "✓ package-lock.json exists"
|
||||||
|
else
|
||||||
|
echo "⚠ package-lock.json does not exist. Creating it..."
|
||||||
|
npm install --package-lock-only
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "========================================="
|
||||||
|
echo "✓ All packages meet the 2-week age requirement"
|
||||||
|
echo "========================================="
|
||||||
|
echo ""
|
||||||
|
echo "To install these packages:"
|
||||||
|
echo " npm ci # Uses exact versions from package-lock.json"
|
||||||
|
echo ""
|
||||||
Reference in New Issue
Block a user