Compare commits

..

16 Commits

Author SHA1 Message Date
pratik
2ee203b012 update styling 2025-10-15 12:02:47 -05:00
pratik
ca91fdfa15 Fix text visibility and table alignment in dark theme
- Improve badge colors for better readability (white text on blue background)
- Fix count badge with bright blue background and white text
- Fix type chip visibility with brighter blue color
- Fix filename cell alignment by using inline-flex instead of flex
- Improve header chip text color visibility
- Increase artifact fetch limit from 25 to 1000 to show all artifacts
- Add proper styling for accent buttons in toolbar

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-15 11:49:00 -05:00
pratik
9733eec2af Add Claude settings to gitignore 2025-10-15 11:44:23 -05:00
pratik
9303f3481b Merge main into f/updates
Resolved conflicts by keeping f/updates changes:
- Keep Angular frontend with dark theme styling
- Keep updated quickstart scripts at root level
- Remove static HTML/JS files (replaced by Angular)
- Keep sim_source_id field implementation
- Merge backend improvements from main

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-15 11:44:01 -05:00
pratik
fbb1dfa67b Integrate dark theme styling from main branch with Angular Material
- Apply dark blue color scheme (#0f172a, #1e293b, #334155) throughout UI
- Update header with blue gradient and Obsidian branding
- Add missing toolbar buttons: Auto-refresh, Seed Data, Search filter
- Implement action buttons (Download, Delete) in artifacts table
- Add client-side search/filtering functionality
- Update app to support sim_source_id field in database
- Move quickstart scripts to repository root for easier access
- Apply dark theme to tables, tabs, and all Material components

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-15 11:42:34 -05:00
2861022ac6 Improve seed data to show clear SIM source grouping
- Guarantee each SIM source has 2-4 artifacts (previously was random)
- Pre-assign artifacts to SIM sources before generation
- 70% of artifacts are grouped, 30% remain ungrouped
- Shuffle assignments to randomize display order
- Makes multi-artifact grouping feature more obvious in demo data

Example output: Each sim_run_* ID now clearly shows 2-4 related artifacts

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-15 09:33:19 -05:00
21347d8c65 Add tags prominence and SIM source grouping features
Database changes:
- Add sim_source_id column to artifacts table for grouping multiple artifacts
- Create Alembic migration (001_add_sim_source_id) for schema update
- Add Alembic env.py for migration support with environment-based DB URLs

API enhancements:
- Add sim_source_id parameter to upload endpoint
- Add sim_source_id filter to query endpoint
- Add new /grouped-by-sim-source endpoint for getting artifacts by group
- Update all API documentation to include sim_source_id

UI improvements:
- Make tags required field and more prominent in upload form
- Add tags display directly in artifacts table (below filename)
- Add SIM Source ID field in upload form with helper text for grouping
- Update table to show sim_source_id (falls back to test_suite if null)
- Tags now displayed as inline badges in main table view

Seed data updates:
- Generate sim_source_id for 70% of artifacts to demonstrate grouping
- Multiple artifacts can share same sim_source_id
- Improved seed data variety with tag combinations

Features:
- Tags are now prominently displayed in both table and detail views
- Multiple artifacts can be grouped by SIM source ID
- Users can filter/query by sim_source_id
- Backward compatible - existing artifacts without sim_source_id still work

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-15 09:30:25 -05:00
6eab60987e Switch PostgreSQL to Alpine-based image for consistency
- Change from postgres:15 to postgres:15-alpine
- Maintains consistency with Alpine-based API container
- Smaller image size and better ARM64 support
- All services now use Alpine or minimal base images

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-15 09:20:11 -05:00
f910c0d67d Switch to Alpine Linux and improve deployment reliability
- Switch from Debian (python:3.11-slim) to Alpine (python:3.11-alpine) for better ARM64 support
- Replace apt-get with apk package manager for lighter, faster builds
- Update package dependencies for Alpine (musl-dev, postgresql-dev, linux-headers)
- Change user creation from useradd to adduser (Alpine syntax)
- Add .gitkeep files to preserve empty alembic directories in git
- Update all quickstart scripts to always rebuild containers with --build flag
- Fixes ARM64 package installation errors on Apple Silicon Macs
- Fixes missing alembic directory errors on fresh clones

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-15 09:13:08 -05:00
17fc9c9b75 Update quickstart scripts to always rebuild containers
- Added --build flag to docker-compose up command
- Ensures latest code changes are always included
- Prevents issues with stale cached images
- Updated all three quickstart scripts (Linux/macOS, Windows batch, PowerShell)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-15 09:06:50 -05:00
85e8776fc3 Revert "Add --fix-missing flag to handle Debian mirror issues"
This reverts commit 1a47ba9369.
2025-10-15 08:59:00 -05:00
1a47ba9369 Add --fix-missing flag to handle Debian mirror issues
- Cleans apt cache before update to avoid stale data
- Adds --fix-missing flag to handle transient mirror synchronization issues
- Fixes hash sum mismatch errors during package installation
- Improves reliability when building on different mirrors

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-15 08:56:03 -05:00
ca86e7a04f Fix ARM64 architecture support in Dockerfile
- Added libpq-dev package for ARM64 PostgreSQL client libraries
- Added --no-install-recommends flag to reduce image size
- Ensures compatibility with Apple Silicon (M1/M2/M3) and ARM servers
- Fixes package installation errors on arm64 architecture

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-15 08:53:39 -05:00
b584cb96bf Add .gitkeep files to preserve alembic directories
- Ensures alembic/ and alembic/versions/ directories exist when cloning
- Fixes error on fresh installations where directories were missing
- Git doesn't track empty directories, .gitkeep solves this

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-15 08:50:05 -05:00
4d9d235111 Add sortable columns and inline search filter
- Implemented sortable table headers with visual indicators (▲▼)
- Click any column to sort ascending/descending
- Sort state persists during auto-refresh
- Added compact inline search filter in toolbar
- Unified search across all columns (Sim Source, Artifacts, Date, Uploaded By)
- Positioned search on right side of toolbar for cleaner layout
- Real-time filtering as you type
- Combined filtering and sorting work seamlessly together

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-15 08:45:31 -05:00
4bc8310070 Rebrand to Obsidian with modern UI and auto-refresh
- Renamed project from "Test Artifact Data Lake" to "Obsidian"
- Updated all branding across README, quickstart scripts, and API
- Implemented dark mode theme with professional color palette
- Simplified table to 4 essential columns (Sim Source, Artifacts, Date, Uploaded By)
- Replaced emoji icons with Lucide SVG icons for better scaling
- Added auto-refresh functionality (5-second intervals, toggleable)
- Enhanced UI with modern flexbox layouts and hover effects
- Updated upload form labels to match new terminology

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-15 08:30:21 -05:00
21 changed files with 560 additions and 93 deletions

View File

@@ -26,7 +26,16 @@
"Bash(ng serve:*)",
"Bash(if exist .angular rmdir /s /q .angular)",
"Bash(dir:*)",
"Bash(tree:*)"
"Bash(tree:*)",
"Bash(git ls-tree:*)",
"mcp__ide__getDiagnostics",
"Read(//c/Users/Pratik/Desktop/code/**)",
"Bash(cat:*)",
"Bash(git add:*)",
"Bash(git commit:*)",
"Bash(git merge:*)",
"Bash(git rm:*)",
"Bash(git checkout:*)"
],
"deny": [],
"ask": []

1
.gitignore vendored
View File

@@ -86,3 +86,4 @@ helm/charts/
tmp/
temp/
*.tmp
.claude/settings.local.json

View File

@@ -1,12 +1,15 @@
FROM python:3.11-slim
FROM python:3.11-alpine
WORKDIR /app
# Install system dependencies
RUN apt-get update && apt-get install -y \
# Install system dependencies for Alpine
# Alpine uses apk instead of apt-get and is lighter/faster
RUN apk add --no-cache \
gcc \
musl-dev \
postgresql-dev \
postgresql-client \
&& rm -rf /var/lib/apt/lists/*
linux-headers
# Copy requirements and install Python dependencies
COPY requirements.txt .
@@ -18,8 +21,8 @@ COPY utils/ ./utils/
COPY alembic/ ./alembic/
COPY alembic.ini .
# Create non-root user
RUN useradd -m -u 1000 appuser && chown -R appuser:appuser /app
# Create non-root user (Alpine uses adduser instead of useradd)
RUN adduser -D -u 1000 appuser && chown -R appuser:appuser /app
USER appuser
# Expose port

View File

@@ -1,4 +1,6 @@
# Test Artifact Data Lake
# Obsidian
**Enterprise Test Artifact Storage**
A lightweight, cloud-native API for storing and querying test artifacts including CSV files, JSON files, binary files, and packet captures (PCAP). Built with FastAPI and supports both AWS S3 and self-hosted MinIO storage backends.

0
alembic/.gitkeep Normal file
View File

84
alembic/env.py Normal file
View File

@@ -0,0 +1,84 @@
from logging.config import fileConfig
import os
from sqlalchemy import engine_from_config
from sqlalchemy import pool
from alembic import context
# Import your models Base
from app.models.artifact import Base
# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config
# Override sqlalchemy.url from environment variable
if os.getenv("DATABASE_URL"):
config.set_main_option("sqlalchemy.url", os.getenv("DATABASE_URL"))
# Interpret the config file for Python logging.
# This line sets up loggers basically.
if config.config_file_name is not None:
fileConfig(config.config_file_name)
# add your model's MetaData object here
# for 'autogenerate' support
target_metadata = Base.metadata
# other values from the config, defined by the needs of env.py,
# can be acquired:
# my_important_option = config.get_main_option("my_important_option")
# ... etc.
def run_migrations_offline() -> None:
"""Run migrations in 'offline' mode.
This configures the context with just a URL
and not an Engine, though an Engine is acceptable
here as well. By skipping the Engine creation
we don't even need a DBAPI to be available.
Calls to context.execute() here emit the given string to the
script output.
"""
url = config.get_main_option("sqlalchemy.url")
context.configure(
url=url,
target_metadata=target_metadata,
literal_binds=True,
dialect_opts={"paramstyle": "named"},
)
with context.begin_transaction():
context.run_migrations()
def run_migrations_online() -> None:
"""Run migrations in 'online' mode.
In this scenario we need to create an Engine
and associate a connection with the context.
"""
connectable = engine_from_config(
config.get_section(config.config_ini_section, {}),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
with connectable.connect() as connection:
context.configure(
connection=connection, target_metadata=target_metadata
)
with context.begin_transaction():
context.run_migrations()
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()

View File

View File

@@ -1,7 +1,7 @@
from fastapi import APIRouter, UploadFile, File, Form, Depends, HTTPException, Query
from fastapi.responses import StreamingResponse
from sqlalchemy.orm import Session
from typing import List, Optional
from typing import List, Optional, Dict
import uuid
import json
import io
@@ -36,6 +36,7 @@ async def upload_artifact(
test_suite: Optional[str] = Form(None),
test_config: Optional[str] = Form(None),
test_result: Optional[str] = Form(None),
sim_source_id: Optional[str] = Form(None),
custom_metadata: Optional[str] = Form(None),
description: Optional[str] = Form(None),
tags: Optional[str] = Form(None),
@@ -51,6 +52,7 @@ async def upload_artifact(
- **test_suite**: Test suite identifier
- **test_config**: JSON string of test configuration
- **test_result**: Test result (pass, fail, skip, error)
- **sim_source_id**: SIM source ID to group multiple artifacts
- **custom_metadata**: JSON string of additional metadata
- **description**: Text description of the artifact
- **tags**: JSON array of tags (as string)
@@ -88,6 +90,7 @@ async def upload_artifact(
test_suite=test_suite,
test_config=test_config_dict,
test_result=test_result,
sim_source_id=sim_source_id,
custom_metadata=metadata_dict,
description=description,
tags=tags_list,
@@ -194,6 +197,7 @@ async def query_artifacts(query: ArtifactQuery, db: Session = Depends(get_db)):
- **test_name**: Filter by test name
- **test_suite**: Filter by test suite
- **test_result**: Filter by test result
- **sim_source_id**: Filter by SIM source ID
- **tags**: Filter by tags (must contain all specified tags)
- **start_date**: Filter by creation date (from)
- **end_date**: Filter by creation date (to)
@@ -212,6 +216,8 @@ async def query_artifacts(query: ArtifactQuery, db: Session = Depends(get_db)):
q = q.filter(Artifact.test_suite == query.test_suite)
if query.test_result:
q = q.filter(Artifact.test_result == query.test_result)
if query.sim_source_id:
q = q.filter(Artifact.sim_source_id == query.sim_source_id)
if query.tags:
for tag in query.tags:
q = q.filter(Artifact.tags.contains([tag]))
@@ -240,3 +246,20 @@ async def list_artifacts(
Artifact.created_at.desc()
).offset(offset).limit(limit).all()
return artifacts
@router.get("/grouped-by-sim-source", response_model=Dict[str, List[ArtifactResponse]])
async def get_artifacts_grouped_by_sim_source(
db: Session = Depends(get_db)
):
"""Get all artifacts grouped by SIM source ID"""
from collections import defaultdict
artifacts = db.query(Artifact).order_by(Artifact.created_at.desc()).all()
grouped = defaultdict(list)
for artifact in artifacts:
sim_source = artifact.sim_source_id or "ungrouped"
grouped[sim_source].append(artifact)
return dict(grouped)

View File

@@ -20,6 +20,7 @@ class Artifact(Base):
test_suite = Column(String(500), index=True)
test_config = Column(JSON)
test_result = Column(String(50), index=True) # pass, fail, skip, error
sim_source_id = Column(String(500), index=True) # SIM source identifier for grouping
# Additional metadata
custom_metadata = Column(JSON)

View File

@@ -8,6 +8,7 @@ class ArtifactCreate(BaseModel):
test_suite: Optional[str] = None
test_config: Optional[Dict[str, Any]] = None
test_result: Optional[str] = None
sim_source_id: Optional[str] = None
custom_metadata: Optional[Dict[str, Any]] = None
description: Optional[str] = None
tags: Optional[List[str]] = None
@@ -26,6 +27,7 @@ class ArtifactResponse(BaseModel):
test_suite: Optional[str] = None
test_config: Optional[Dict[str, Any]] = None
test_result: Optional[str] = None
sim_source_id: Optional[str] = None
custom_metadata: Optional[Dict[str, Any]] = None
description: Optional[str] = None
tags: Optional[List[str]] = None
@@ -44,6 +46,7 @@ class ArtifactQuery(BaseModel):
test_name: Optional[str] = None
test_suite: Optional[str] = None
test_result: Optional[str] = None
sim_source_id: Optional[str] = None
tags: Optional[List[str]] = None
start_date: Optional[datetime] = None
end_date: Optional[datetime] = None

View File

@@ -2,7 +2,7 @@ version: '3.8'
services:
postgres:
image: postgres:15
image: postgres:15-alpine
environment:
POSTGRES_USER: user
POSTGRES_PASSWORD: password

View File

@@ -1,18 +1,18 @@
<div class="app-container">
<!-- Material Toolbar Header -->
<mat-toolbar color="primary" class="app-toolbar">
<mat-icon class="app-icon">storage</mat-icon>
<span class="app-title">{{ title }}</span>
<mat-icon class="app-icon">diamond</mat-icon>
<span class="app-title">◆ Obsidian</span>
<span class="spacer"></span>
<div class="header-info" *ngIf="apiInfo">
<mat-chip-set>
<mat-chip>
<mat-icon matChipAvatar>settings</mat-icon>
{{ apiInfo.deployment_mode }}
Mode: {{ apiInfo.deployment_mode }}
</mat-chip>
<mat-chip>
<mat-icon matChipAvatar>folder</mat-icon>
{{ apiInfo.storage_backend }}
Storage: {{ apiInfo.storage_backend }}
</mat-chip>
</mat-chip-set>
</div>
@@ -24,18 +24,30 @@
<mat-tab label="Artifacts">
<ng-template matTabContent>
<div class="tab-content-wrapper">
<div class="content-header">
<h2>
<mat-icon>storage</mat-icon>
Artifacts ({{ artifacts.length }})
</h2>
<div class="toolbar">
<button mat-raised-button color="primary" (click)="loadArtifacts()">
<mat-icon>refresh</mat-icon>
Refresh
</button>
<button mat-raised-button [color]="autoRefreshEnabled ? 'accent' : ''" (click)="toggleAutoRefresh()">
Auto-refresh: {{ autoRefreshEnabled ? 'ON' : 'OFF' }}
</button>
<button mat-raised-button (click)="generateSeedData()">
<mat-icon>auto_awesome</mat-icon>
Generate Seed Data
</button>
<span class="count-badge">{{ artifacts.length }} artifacts</span>
<mat-form-field appearance="outline" class="filter-search">
<mat-label>Search</mat-label>
<input matInput [(ngModel)]="searchTerm" (input)="filterTable()" placeholder="Search...">
<mat-icon matPrefix>search</mat-icon>
<button mat-icon-button matSuffix *ngIf="searchTerm" (click)="clearSearch()">
<mat-icon>close</mat-icon>
</button>
</mat-form-field>
</div>
<table mat-table [dataSource]="artifacts" class="artifacts-table mat-elevation-z4">
<table mat-table [dataSource]="filteredArtifacts" class="artifacts-table mat-elevation-z4">
<!-- ID Column -->
<ng-container matColumnDef="id">
<th mat-header-cell *matHeaderCellDef>ID</th>
@@ -82,6 +94,21 @@
</td>
</ng-container>
<!-- Actions Column -->
<ng-container matColumnDef="actions">
<th mat-header-cell *matHeaderCellDef>Actions</th>
<td mat-cell *matCellDef="let artifact">
<div class="action-buttons">
<button mat-icon-button (click)="downloadArtifact(artifact)" matTooltip="Download">
<mat-icon>download</mat-icon>
</button>
<button mat-icon-button (click)="deleteArtifact(artifact)" matTooltip="Delete" color="warn">
<mat-icon>delete</mat-icon>
</button>
</div>
</td>
</ng-container>
<tr mat-header-row *matHeaderRowDef="displayedColumns"></tr>
<tr mat-row *matRowDef="let row; columns: displayedColumns;"></tr>
</table>

View File

@@ -2,14 +2,18 @@
display: flex;
flex-direction: column;
height: 100vh;
background: #f5f5f5;
background: #1e293b;
border-radius: 12px;
box-shadow: 0 20px 60px rgba(0, 0, 0, 0.5);
overflow: hidden;
}
.app-toolbar {
position: sticky;
top: 0;
z-index: 10;
box-shadow: 0 2px 8px rgba(0,0,0,0.15);
background: linear-gradient(135deg, #1e3a8a 0%, #4338ca 100%) !important;
box-shadow: 0 2px 8px rgba(0,0,0,0.3);
}
.app-icon {
@@ -27,34 +31,45 @@
}
.header-info {
mat-chip-set {
mat-chip {
background-color: rgba(255, 255, 255, 0.2) !important;
color: white !important;
::ng-deep {
mat-chip-set {
mat-chip {
background-color: rgba(255, 255, 255, 0.2) !important;
mat-icon {
color: white !important;
.mdc-evolution-chip__action {
color: white !important;
}
.mdc-evolution-chip__text-label {
color: white !important;
}
mat-icon {
color: white !important;
}
}
}
}
}
// Tab Group Styling
// Tab Group Styling - Dark Theme
.main-tabs {
flex: 1;
display: flex;
flex-direction: column;
background: white;
background: #1e293b;
::ng-deep {
.mat-mdc-tab-body-wrapper {
flex: 1;
padding: 0;
background: #1e293b;
}
.mat-mdc-tab-header {
background: white;
box-shadow: 0 2px 4px rgba(0,0,0,0.08);
background: #0f172a;
box-shadow: 0 2px 4px rgba(0,0,0,0.3);
border-bottom: 2px solid #334155;
}
.mat-mdc-tab-labels {
@@ -67,28 +82,54 @@
font-size: 14px;
font-weight: 500;
letter-spacing: 0.5px;
text-transform: uppercase;
.mdc-tab__text-label {
color: #cbd5e1;
}
&:hover {
background: #1e293b;
.mdc-tab__text-label {
color: #e2e8f0;
}
}
}
.mat-mdc-tab.mdc-tab--active {
.mdc-tab__text-label {
color: #60a5fa;
font-weight: 600;
}
}
.mat-mdc-tab-indicator {
.mdc-tab-indicator__content--underline {
background-color: #60a5fa;
height: 3px;
}
}
}
}
// Tab Content Wrapper
// Tab Content Wrapper - Dark Theme
.tab-content-wrapper {
padding: 24px;
padding: 30px;
max-width: 1400px;
margin: 0 auto;
width: 100%;
min-height: calc(100vh - 180px);
background: #1e293b;
}
// Content Header
// Content Header - Dark Theme
.content-header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 24px;
padding-bottom: 16px;
border-bottom: 2px solid #e0e0e0;
margin-bottom: 20px;
flex-wrap: wrap;
gap: 10px;
h2 {
display: flex;
@@ -97,10 +138,10 @@
margin: 0;
font-size: 24px;
font-weight: 500;
color: #333;
color: #e2e8f0;
mat-icon {
color: #3f51b5;
color: #60a5fa;
font-size: 28px;
width: 28px;
height: 28px;
@@ -114,90 +155,214 @@
}
}
// Artifacts Table Styling
// Toolbar styling
.toolbar {
display: flex;
gap: 10px;
margin-bottom: 20px;
align-items: center;
flex-wrap: wrap;
// Ensure buttons have proper styling
button {
&[color="accent"] {
background-color: #10b981 !important;
color: white !important;
}
}
}
.count-badge {
background: #3b82f6;
color: #ffffff;
padding: 8px 16px;
border-radius: 20px;
font-size: 13px;
font-weight: 600;
margin-left: auto;
}
// Filter Search Styling
.filter-search {
min-width: 250px;
::ng-deep {
.mat-mdc-text-field-wrapper {
background: #0f172a;
border-radius: 6px;
}
.mat-mdc-form-field-input-control {
color: #e2e8f0;
}
.mat-mdc-form-field-label {
color: #94a3b8;
}
.mdc-notched-outline__leading,
.mdc-notched-outline__notch,
.mdc-notched-outline__trailing {
border-color: #334155 !important;
}
.mat-mdc-form-field:hover .mdc-notched-outline__leading,
.mat-mdc-form-field:hover .mdc-notched-outline__notch,
.mat-mdc-form-field:hover .mdc-notched-outline__trailing {
border-color: #60a5fa !important;
}
.mat-mdc-form-field-icon-prefix,
.mat-mdc-form-field-icon-suffix {
color: #64748b;
}
}
}
// Action buttons styling
.action-buttons {
display: flex;
gap: 4px;
button {
color: #94a3b8;
&:hover {
color: #e2e8f0;
background: #334155;
}
}
}
// Artifacts Table Styling - Dark Theme
.artifacts-table {
width: 100%;
background: white;
background: #0f172a;
border-radius: 8px;
overflow: hidden;
border: 1px solid #334155;
th.mat-mdc-header-cell {
background: #f8f9fa;
color: #333;
background: #1e293b;
color: #94a3b8;
font-weight: 600;
font-size: 14px;
font-size: 12px;
text-transform: uppercase;
letter-spacing: 0.5px;
padding: 16px 12px;
border-bottom: 2px solid #e0e0e0;
padding: 14px 12px;
border-bottom: 2px solid #334155;
}
td.mat-mdc-cell {
padding: 16px 12px;
font-size: 14px;
color: #555;
border-bottom: 1px solid #f0f0f0;
color: #cbd5e1;
border-bottom: 1px solid #1e293b;
}
tr.mat-mdc-row {
transition: background-color 0.2s ease;
background: #0f172a;
&:hover {
background-color: #f8f9fa;
background-color: #1e293b;
}
}
.filename-cell {
display: flex;
align-items: center;
gap: 8px;
td.filename-cell {
font-weight: 500;
.file-icon {
color: #3f51b5;
mat-icon {
color: #60a5fa;
font-size: 20px;
width: 20px;
height: 20px;
vertical-align: middle;
margin-right: 8px;
}
}
.type-chip {
background-color: #e3f2fd !important;
color: #1976d2 !important;
font-weight: 500;
font-size: 12px;
background-color: #3b82f6 !important;
color: #ffffff !important;
font-weight: 600;
font-size: 11px;
padding: 4px 12px;
height: 28px;
height: auto;
}
.text-muted {
color: #999;
color: #64748b;
}
}
// Result Chips
.result-pass {
background-color: #e8f5e9 !important;
color: #2e7d32 !important;
font-weight: 600;
// Result Chips - Material Design style
mat-chip.result-pass {
--mdc-chip-elevated-container-color: #4caf50 !important;
--mdc-chip-label-text-color: #ffffff !important;
background-color: #4caf50 !important;
.mdc-evolution-chip__action {
color: #ffffff !important;
}
.mdc-evolution-chip__text-label {
color: #ffffff !important;
font-weight: 500;
font-size: 12px;
text-transform: uppercase;
}
}
.result-fail {
background-color: #ffebee !important;
color: #c62828 !important;
font-weight: 600;
mat-chip.result-fail {
--mdc-chip-elevated-container-color: #f44336 !important;
--mdc-chip-label-text-color: #ffffff !important;
background-color: #f44336 !important;
.mdc-evolution-chip__action {
color: #ffffff !important;
}
.mdc-evolution-chip__text-label {
color: #ffffff !important;
font-weight: 500;
font-size: 12px;
text-transform: uppercase;
}
}
.result-skip {
background-color: #fff3e0 !important;
color: #ef6c00 !important;
font-weight: 600;
mat-chip.result-skip {
--mdc-chip-elevated-container-color: #ff9800 !important;
--mdc-chip-label-text-color: #ffffff !important;
background-color: #ff9800 !important;
.mdc-evolution-chip__action {
color: #ffffff !important;
}
.mdc-evolution-chip__text-label {
color: #ffffff !important;
font-weight: 500;
font-size: 12px;
text-transform: uppercase;
}
}
.result-error {
background-color: #fce4ec !important;
color: #c2185b !important;
font-weight: 600;
mat-chip.result-error {
--mdc-chip-elevated-container-color: #e91e63 !important;
--mdc-chip-label-text-color: #ffffff !important;
background-color: #e91e63 !important;
.mdc-evolution-chip__action {
color: #ffffff !important;
}
.mdc-evolution-chip__text-label {
color: #ffffff !important;
font-weight: 500;
font-size: 12px;
text-transform: uppercase;
}
}
// Responsive Design

View File

@@ -1,11 +1,15 @@
import { Component, OnInit } from '@angular/core';
import { Component, OnInit, OnDestroy } from '@angular/core';
import { CommonModule } from '@angular/common';
import { FormsModule } from '@angular/forms';
import { MatToolbarModule } from '@angular/material/toolbar';
import { MatTableModule } from '@angular/material/table';
import { MatTabsModule } from '@angular/material/tabs';
import { MatChipsModule } from '@angular/material/chips';
import { MatIconModule } from '@angular/material/icon';
import { MatButtonModule } from '@angular/material/button';
import { MatInputModule } from '@angular/material/input';
import { MatFormFieldModule } from '@angular/material/form-field';
import { MatTooltipModule } from '@angular/material/tooltip';
import { UploadFormComponent } from './components/upload-form/upload-form.component';
import { QueryFormComponent } from './components/query-form/query-form.component';
import { ApiService } from './services/api.service';
@@ -16,24 +20,32 @@ import { ApiInfo, Artifact } from './models/artifact.interface';
selector: 'app-root',
imports: [
CommonModule,
FormsModule,
MatToolbarModule,
MatTableModule,
MatTabsModule,
MatChipsModule,
MatIconModule,
MatButtonModule,
MatInputModule,
MatFormFieldModule,
MatTooltipModule,
UploadFormComponent,
QueryFormComponent
],
templateUrl: './app.component.html',
styleUrl: './app.component.scss'
})
export class AppComponent implements OnInit {
title = 'Test Artifact Data Lake';
export class AppComponent implements OnInit, OnDestroy {
title = 'Obsidian - Test Artifact Data Lake';
apiInfo: ApiInfo | null = null;
artifacts: Artifact[] = [];
displayedColumns: string[] = ['id', 'filename', 'file_type', 'file_size', 'test_name', 'test_result'];
filteredArtifacts: Artifact[] = [];
displayedColumns: string[] = ['id', 'filename', 'file_type', 'file_size', 'test_name', 'test_result', 'actions'];
selectedTabIndex = 0;
searchTerm: string = '';
autoRefreshEnabled: boolean = true;
private autoRefreshInterval: any;
constructor(
private apiService: ApiService,
@@ -43,6 +55,11 @@ export class AppComponent implements OnInit {
ngOnInit(): void {
this.loadApiInfo();
this.loadArtifacts();
this.startAutoRefresh();
}
ngOnDestroy(): void {
this.stopAutoRefresh();
}
loadApiInfo(): void {
@@ -61,6 +78,7 @@ export class AppComponent implements OnInit {
next: (artifacts) => {
console.log('Loaded artifacts:', artifacts.length);
this.artifacts = artifacts;
this.filterTable();
},
error: (error) => {
console.error('Error loading artifacts:', error);
@@ -68,18 +86,122 @@ export class AppComponent implements OnInit {
});
}
filterTable(): void {
if (!this.searchTerm) {
this.filteredArtifacts = this.artifacts;
return;
}
const term = this.searchTerm.toLowerCase();
this.filteredArtifacts = this.artifacts.filter(artifact => {
return (
artifact.filename?.toLowerCase().includes(term) ||
artifact.test_name?.toLowerCase().includes(term) ||
artifact.test_suite?.toLowerCase().includes(term) ||
artifact.file_type?.toLowerCase().includes(term)
);
});
}
clearSearch(): void {
this.searchTerm = '';
this.filterTable();
}
toggleAutoRefresh(): void {
this.autoRefreshEnabled = !this.autoRefreshEnabled;
if (this.autoRefreshEnabled) {
this.startAutoRefresh();
} else {
this.stopAutoRefresh();
}
}
startAutoRefresh(): void {
this.stopAutoRefresh();
if (this.autoRefreshEnabled) {
this.autoRefreshInterval = setInterval(() => {
if (this.selectedTabIndex === 0) {
this.loadArtifacts();
}
}, 5000);
}
}
stopAutoRefresh(): void {
if (this.autoRefreshInterval) {
clearInterval(this.autoRefreshInterval);
this.autoRefreshInterval = null;
}
}
generateSeedData(): void {
const count = prompt('How many artifacts to generate? (1-100)', '10');
if (!count) return;
const num = parseInt(count);
if (isNaN(num) || num < 1 || num > 100) {
alert('Please enter a number between 1 and 100');
return;
}
this.artifactService.generateSeedData(num).subscribe({
next: (result: any) => {
alert(result.message || `Successfully generated ${num} artifacts`);
this.loadArtifacts();
},
error: (error) => {
alert('Error generating seed data: ' + error.message);
}
});
}
downloadArtifact(artifact: Artifact): void {
this.artifactService.downloadArtifact(artifact.id).subscribe({
next: (blob) => {
const url = window.URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = artifact.filename;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
document.body.removeChild(a);
},
error: (error) => {
alert('Error downloading artifact: ' + error.message);
}
});
}
deleteArtifact(artifact: Artifact): void {
if (!confirm(`Are you sure you want to delete "${artifact.filename}"? This cannot be undone.`)) {
return;
}
this.artifactService.deleteArtifact(artifact.id).subscribe({
next: () => {
alert('Artifact deleted successfully');
this.loadArtifacts();
},
error: (error) => {
alert('Error deleting artifact: ' + error.message);
}
});
}
onUploadSuccess(): void {
this.loadArtifacts();
this.selectedTabIndex = 0; // Switch back to artifacts tab
this.selectedTabIndex = 0;
}
onQueryResults(artifacts: Artifact[]): void {
this.artifacts = artifacts;
this.selectedTabIndex = 0; // Switch to artifacts tab to show results
this.filterTable();
this.selectedTabIndex = 0;
}
onFiltersChange(filters: any): void {
// Filters will be handled by the query component
console.log('Filters changed:', filters);
}

View File

@@ -14,7 +14,7 @@ export class ArtifactService {
constructor(private http: HttpClient) { }
getArtifacts(limit: number = 25, offset: number = 0): Observable<Artifact[]> {
getArtifacts(limit: number = 1000, offset: number = 0): Observable<Artifact[]> {
const params = new HttpParams()
.set('limit', limit.toString())
.set('offset', offset.toString());

View File

@@ -1,4 +1,4 @@
/* Global styles for the Test Artifact Data Lake Angular app with Material Design */
/* Global styles for Obsidian - Dark Theme inspired from main branch */
* {
margin: 0;
@@ -7,10 +7,11 @@
}
body {
font-family: Roboto, "Helvetica Neue", sans-serif;
background: linear-gradient(135deg, #3f51b5 0%, #9c27b0 100%);
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, sans-serif;
background: #0f172a;
min-height: 100vh;
padding: 20px;
color: #e2e8f0;
}
.container {

View File

@@ -129,7 +129,7 @@ def generate_pcap_content() -> bytes:
return bytes(pcap_header)
def create_artifact_data(index: int) -> Dict[str, Any]:
def create_artifact_data(index: int, sim_source_id: str = None) -> Dict[str, Any]:
"""Generate metadata for an artifact"""
test_name = random.choice(TEST_NAMES)
test_suite = random.choice(TEST_SUITES)
@@ -164,6 +164,7 @@ def create_artifact_data(index: int) -> Dict[str, Any]:
"test_name": test_name,
"test_suite": test_suite,
"test_result": test_result,
"sim_source_id": sim_source_id,
"tags": artifact_tags,
"test_config": test_config,
"custom_metadata": custom_metadata,
@@ -265,6 +266,27 @@ async def generate_seed_data(num_artifacts: int = 50) -> List[int]:
print(f"Deployment mode: {settings.deployment_mode}")
print(f"Storage backend: {settings.storage_backend}")
# Generate SIM source IDs - each source will have 2-4 artifacts
num_sim_sources = max(num_artifacts // 3, 1)
sim_sources = [f"sim_run_{uuid.uuid4().hex[:8]}" for _ in range(num_sim_sources)]
# Pre-assign artifacts to SIM sources to ensure grouping
sim_source_assignments = []
for sim_source in sim_sources:
# Each SIM source gets 2-4 artifacts
num_artifacts_for_source = random.randint(2, 4)
sim_source_assignments.extend([sim_source] * num_artifacts_for_source)
# Pad remaining artifacts with None (ungrouped) or random sources
while len(sim_source_assignments) < num_artifacts:
if random.random() < 0.3: # 30% ungrouped
sim_source_assignments.append(None)
else:
sim_source_assignments.append(random.choice(sim_sources))
# Shuffle to randomize order
random.shuffle(sim_source_assignments)
for i in range(num_artifacts):
# Randomly choose file type
file_type_choice = random.choice(['csv', 'json', 'binary', 'pcap'])
@@ -289,8 +311,11 @@ async def generate_seed_data(num_artifacts: int = 50) -> List[int]:
# Upload to storage
storage_path = await upload_artifact_to_storage(content, filename)
# Get pre-assigned SIM source ID for this artifact
sim_source_id = sim_source_assignments[i]
# Generate metadata
artifact_data = create_artifact_data(i)
artifact_data = create_artifact_data(i, sim_source_id)
# Create database record
artifact = Artifact(
@@ -303,6 +328,7 @@ async def generate_seed_data(num_artifacts: int = 50) -> List[int]:
test_suite=artifact_data["test_suite"],
test_config=artifact_data["test_config"],
test_result=artifact_data["test_result"],
sim_source_id=artifact_data["sim_source_id"],
custom_metadata=artifact_data["custom_metadata"],
description=artifact_data["description"],
tags=artifact_data["tags"],