jeffemmett
/
infinite-agents-public
mirror of https://github.com/Jeff-Emmett/infinite-agents-public.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
infinite-agents-public/infinite_variants/infinite_variant_1/ARCHITECTURE.md

803 lines
22 KiB
Markdown
Raw Blame History

# Architecture Documentation
Technical architecture of the Cross-Iteration Pattern Synthesis System.
## System Overview
```
┌─────────────────────────────────────────────────────────────────┐
│ ORCHESTRATOR AGENT │
│ (infinite-synthesis.md) │
└─────────────────────────────────────────────────────────────────┘
│
├─── Wave 1: Cold Start
│ │
│ ├─> Sub-Agent 1 ─> Iteration 1
│ ├─> Sub-Agent 2 ─> Iteration 2
│ ├─> Sub-Agent 3 ─> Iteration 3
│ ├─> Sub-Agent 4 ─> Iteration 4
│ └─> Sub-Agent 5 ─> Iteration 5
│
├─── Pattern Extraction
│ │
│ └─> Extract Patterns Agent
│ └─> Pattern Library v1.0
│
├─── Wave 2: Pattern-Guided
│ │
│ ├─> Sub-Agent 6 (+ patterns) ─> Iteration 6
│ ├─> Sub-Agent 7 (+ patterns) ─> Iteration 7
│ ├─> Sub-Agent 8 (+ patterns) ─> Iteration 8
│ ├─> Sub-Agent 9 (+ patterns) ─> Iteration 9
│ └─> Sub-Agent 10 (+ patterns) ─> Iteration 10
│
├─── Pattern Refinement
│ │
│ └─> Extract Patterns Agent
│ └─> Pattern Library v1.1
│
└─── Wave 3+ (Continuous Learning)
└─> ... (repeat until count reached)
```
## Core Components
### 1. Orchestrator Agent
**File**: `.claude/commands/infinite-synthesis.md`
**Responsibilities**:
- Parse command arguments (spec, output dir, count, pattern library path)
- Calculate wave parameters (number of waves, iterations per wave)
- Coordinate wave execution
- Trigger pattern extraction between waves
- Manage context budget
- Generate final report
**State Management**:
```javascript
{
total_count: 20,
waves: 4,
wave_size: 5,
current_wave: 1,
pattern_library_version: "1.0",
iterations_generated: [],
quality_metrics: []
}
```
**Key Algorithms**:
```python
# Wave calculation
def calculate_waves(count):
if count == "infinite":
return infinite_waves, 5
elif count <= 5:
return 1, count
elif count <= 15:
return 2, count // 2
else:
return count // 5, 5
# Pattern extraction trigger
def should_extract_patterns(current_wave, total_waves):
# Extract after every wave except the last
return current_wave < total_waves
```
### 2. Sub-Agent System
**Created via**: Task tool
**Context Provided**:
```markdown
SPECIFICATION:
{Full spec content}
EXISTING ITERATIONS:
{List of already generated files}
PATTERN LIBRARY (Wave 2+ only):
{3-5 most relevant patterns}
REQUIREMENTS:
- Generate unique iteration
- Follow specification
- Incorporate patterns (if provided)
- Add novel innovation
- Maintain quality standards
OUTPUT:
Save to: {output_path}
```
**Execution Model**:
- Parallel execution (5 sub-agents at a time)
- Independent context (each agent has full spec + patterns)
- Synchronization point: All agents complete before pattern extraction
### 3. Pattern Extraction Agent
**File**: `.claude/commands/extract-patterns.md`
**Responsibilities**:
- Read all iteration files
- Score iterations across dimensions (functionality, quality, innovation, etc.)
- Identify top 20% per category
- Extract patterns with examples
- Build/update pattern library JSON
- Validate library structure
- Generate extraction report
**Scoring Dimensions**:
```javascript
{
functionality: 0-10, // Does it work as specified?
visual_appeal: 0-10, // Aesthetics and UX
code_quality: 0-10, // Readability, organization
innovation: 0-10, // Novel ideas and creativity
documentation: 0-10, // Comments and explanations
robustness: 0-10 // Error handling, edge cases
}
overall_score = average(dimensions)
```
**Pattern Selection Algorithm**:
```python
def extract_patterns(iterations, category, count=5):
# 1. Score all iterations for this category
scored = [(iteration, score_for_category(iteration, category))
for iteration in iterations]
# 2. Sort by score (descending)
scored.sort(key=lambda x: x[1], reverse=True)
# 3. Take top 20%
top_20_percent = scored[:len(scored)//5]
# 4. Select diverse patterns
patterns = []
for iteration, score in top_20_percent:
pattern = extract_pattern_from(iteration, category)
if is_diverse_from(pattern, patterns):
patterns.append(pattern)
if len(patterns) >= count:
break
return patterns
```
### 4. Pattern Library
**File**: `pattern_library/patterns.json`
**Schema**:
```json
{
"version": "semver",
"last_updated": "ISO 8601 timestamp",
"total_iterations_analyzed": "integer",
"analysis_depth": "quick|deep",
"patterns": {
"structural": [/* 3-5 pattern objects */],
"content": [/* 3-5 pattern objects */],
"innovation": [/* 3-5 pattern objects */],
"quality": [/* 3-5 pattern objects */]
},
"metadata": {
"extraction_date": "ISO 8601",
"source_directory": "path",
"patterns_extracted": "count",
"avg_quality_score": "float"
}
}
```
**Pattern Object Schema**:
```json
{
"name": "string (short, descriptive)",
"description": "string (1-2 sentences)",
"example_file": "string (path to exemplary iteration)",
"key_characteristics": ["array", "of", "defining", "traits"],
"success_metrics": "string (specific, measurable)",
"code_snippet": "string (5-15 lines representative code)"
}
```
**Update Strategy**:
```python
def update_pattern_library(old_library, new_iterations):
# Extract patterns from new iterations only
new_patterns = extract_all_patterns(new_iterations)
# Merge with existing patterns
for category in categories:
# Combine old and new patterns
all_patterns = old_library[category] + new_patterns[category]
# Rank by effectiveness
ranked = rank_patterns(all_patterns)
# Keep top 5 (or 3 for quick mode)
old_library[category] = ranked[:5]
# Increment version
old_library["version"] = increment_version(old_library["version"])
return old_library
```
### 5. Analysis Agent
**File**: `.claude/commands/analyze-patterns.md`
**Responsibilities**:
- Load pattern library
- Categorize iterations (pre-pattern vs post-pattern)
- Calculate adoption rate
- Compare quality metrics
- Rank pattern effectiveness
- Generate analysis report
**Metrics Calculated**:
```javascript
{
// Adoption metrics
pattern_adoption_rate: percent,
avg_patterns_per_iteration: float,
most_adopted_pattern: pattern_name,
least_adopted_pattern: pattern_name,
// Quality metrics
pre_pattern_quality: float,
post_pattern_quality: float,
quality_improvement: percent,
consistency_improvement: percent,
// Innovation metrics
pre_pattern_innovations: count,
post_pattern_innovations: count,
innovation_preservation: percent,
// Pattern effectiveness
pattern_rankings: [
{pattern: name, adoption: percent, impact: float}
]
}
```
### 6. Validation System
**File**: `validators/check_patterns.sh`
**Validations Performed**:
```bash
# 1. JSON Syntax
jq empty pattern_library.json
# 2. Required Fields
for field in version last_updated patterns metadata
check_exists(field)
# 3. Pattern Categories
for category in structural content innovation quality
check_exists(patterns[category])
check_count(patterns[category], 3-5)
# 4. Pattern Objects
for pattern in all_patterns
check_fields(name, description, example_file,
key_characteristics, success_metrics, code_snippet)
# 5. Pattern Quality
calculate_snippet_coverage()
calculate_metrics_coverage()
# 6. Consistency Checks
check_no_duplicate_names()
check_version_incremented()
```
## Data Flow
### Wave 1: Cold Start Generation
```
User Command
│
├─> Parse Arguments
│ └─> spec_file, output_dir, count=5
│
├─> Read Specification
│ └─> Load spec content
│
├─> Create Sub-Agents (x5)
│ │
│ ├─> Sub-Agent 1: {spec, existing_iterations=[]}
│ ├─> Sub-Agent 2: {spec, existing_iterations=[iter_1]}
│ ├─> Sub-Agent 3: {spec, existing_iterations=[iter_1, iter_2]}
│ ├─> Sub-Agent 4: {spec, existing_iterations=[iter_1..3]}
│ └─> Sub-Agent 5: {spec, existing_iterations=[iter_1..4]}
│
├─> Execute in Parallel
│ └─> Wait for all to complete
│
├─> Collect Outputs
│ └─> [iteration_1..5.html]
│
└─> Trigger Pattern Extraction
└─> See Pattern Extraction Flow
```
### Pattern Extraction Flow
```
Extract Patterns Command
│
├─> Read All Iterations
│ └─> [iteration_1..5.html]
│
├─> Score Each Iteration
│ │
│ ├─> Structural Score
│ ├─> Content Score
│ ├─> Innovation Score
│ └─> Quality Score
│
├─> Identify Top 20% per Category
│ │
│ ├─> Structural: [iter_3, iter_5]
│ ├─> Content: [iter_2, iter_5]
│ ├─> Innovation: [iter_1, iter_4]
│ └─> Quality: [iter_3, iter_4]
│
├─> Extract Pattern Objects
│ │
│ ├─> For each top iteration:
│ │ ├─> Analyze code structure
│ │ ├─> Extract key characteristics
│ │ ├─> Capture code snippet
│ │ └─> Document success metrics
│ │
│ └─> Select 3-5 most diverse patterns per category
│
├─> Build Pattern Library JSON
│ │
│ └─> {
│ version: "1.0",
│ patterns: {
│ structural: [pattern1, pattern2, pattern3],
│ content: [pattern1, pattern2, pattern3],
│ ...
│ }
│ }
│
├─> Validate Pattern Library
│ └─> Run check_patterns.sh
│
├─> Save to File
│ └─> pattern_library/patterns.json
│
└─> Generate Report
└─> Pattern extraction summary
```
### Wave 2+: Pattern-Guided Generation
```
Continue Generation (Wave 2)
│
├─> Load Pattern Library
│ └─> pattern_library/patterns.json v1.0
│
├─> Create Sub-Agents (x5)
│ │
│ ├─> Sub-Agent 6:
│ │ ├─> spec
│ │ ├─> existing_iterations=[iter_1..5]
│ │ └─> relevant_patterns=[
│ │ structural_pattern_1,
│ │ content_pattern_1,
│ │ quality_pattern_1
│ │ ]
│ │
│ ├─> Sub-Agent 7: (similar context + patterns)
│ └─> ... (Sub-Agents 8-10)
│
├─> Execute in Parallel
│ └─> Sub-agents incorporate pattern examples
│
├─> Collect Outputs
│ └─> [iteration_6..10.html]
│
├─> Extract Patterns from ALL iterations
│ │
│ ├─> Analyze [iteration_1..10.html]
│ ├─> Extract new patterns from iterations 6-10
│ ├─> Merge with existing patterns
│ ├─> Keep top 5 per category
│ └─> Increment version to v1.1
│
└─> Continue to Wave 3 if count allows
```
## Multi-Shot Prompting Integration
### How Patterns Serve as Examples
When a sub-agent receives pattern context:
```markdown
PATTERN CONTEXT PROVIDED:
### Structural Pattern: Modular Three-Layer Architecture
**Description**: Separates data, rendering logic, and interaction handlers
**Why This Works**: Readability 9.5/10, easy to test, modifications don't cascade
**Example Code**:
```javascript
// DATA LAYER
const dataset = {
values: [...],
validate() { return this.values.length > 0; }
};
// VIEW LAYER
const renderer = {
render(data) { /* D3 rendering */ }
};
// CONTROLLER LAYER
const controller = {
onNodeClick(e) { /* interaction logic */ }
};
```
**Key Characteristics**:
- Clear layer boundaries with comments
- Data validation methods on data objects
- Pure rendering functions (no business logic)
- Event handlers isolated in controller
---
[2-4 more patterns provided...]
YOUR TASK:
Study these patterns. Understand WHY they work (success metrics).
Apply their principles to your iteration.
Add your own innovation beyond these examples.
```
### Pattern as Multi-Shot Example
This is textbook multi-shot prompting:
1. **Concrete Example**: Actual code, not just description
2. **Success Context**: "Why This Works" explains effectiveness
3. **Multiple Examples**: 3-5 patterns provide diversity
4. **Clear Structure**: Consistent format makes patterns easy to parse
5. **Transferable**: Characteristics list shows how to adapt
Research shows this approach (3-5 concrete examples with success context) maximizes consistency while preserving creativity.
## Context Budget Management
### Context Allocation
```
Total Context Budget: ~200K tokens
Allocation per Wave:
├─ Specification: ~2K tokens
├─ Pattern Library: ~3K tokens (grows slightly over time)
├─ Sub-Agent Context (x5): ~15K tokens total
│ ├─ Spec: 2K
│ ├─ Patterns: 3K
│ ├─ Existing iterations list: 500 tokens
│ └─ Task instructions: 1K
├─ Pattern Extraction: ~5K tokens
└─ Orchestrator Logic: ~2K tokens
Per Wave Total: ~27K tokens
Maximum Waves: 200K / 27K ≈ 7 waves (35 iterations)
```
### Context Optimization Strategies
1. **Pattern Library Size Cap**: Max 5 patterns per category (3 for "quick" mode)
2. **Iteration List Compression**: Only file names, not content
3. **Selective Pattern Provision**: Provide 3-5 most relevant patterns, not all
4. **Summary vs Full Content**: Pattern extraction works with summaries
5. **Garbage Collection**: Remove obsolete patterns as better ones emerge
### Infinite Mode Termination
```python
def should_continue_infinite(context_usage):
# Stop if context usage exceeds 80% of budget
if context_usage > 0.8 * CONTEXT_BUDGET:
return False, "Context budget limit approaching"
# Stop if pattern library isn't improving
if library_unchanged_for_N_waves(3):
return False, "Pattern library converged"
# Stop if quality plateaued
if quality_unchanged_for_N_waves(5):
return False, "Quality plateau reached"
return True, "Continue generation"
```
## Error Handling
### Orchestrator Level
```python
try:
# Execute wave
iterations = execute_wave(wave_num)
except SubAgentFailure as e:
# Log error, continue with successful iterations
log_error(f"Sub-agent {e.agent_id} failed: {e.message}")
# Optionally retry failed iteration
if should_retry(e):
retry_iteration(e.iteration_num)
```
### Pattern Extraction Level
```python
try:
# Extract patterns
patterns = extract_patterns(iterations)
except ExtractionFailure as e:
# Log warning, use previous pattern library
log_warning(f"Pattern extraction failed: {e.message}")
log_info("Continuing with existing pattern library")
patterns = load_previous_library()
```
### Sub-Agent Level
```python
try:
# Generate iteration
output = generate_iteration(spec, patterns)
validate_output(output)
except GenerationFailure as e:
# Report to orchestrator
return Error(f"Failed to generate iteration: {e.message}")
```
### Validation Level
```bash
# Validator returns non-zero exit code on failure
if ! ./validators/check_patterns.sh "$PATTERN_LIB"; then
echo "Pattern library validation failed"
echo "Fix errors before continuing"
exit 1
fi
```
## Performance Considerations
### Parallel Execution
Sub-agents execute in parallel:
```
Wave of 5 iterations:
Traditional Sequential:
Agent 1 ────> (2 min)
Agent 2 ────> (2 min)
Agent 3 ────> (2 min)
Agent 4 ────> (2 min)
Agent 5 ────> (2 min)
Total: 10 minutes
Parallel Execution:
Agent 1 ────> (2 min)
Agent 2 ────> (2 min)
Agent 3 ────> (2 min)
Agent 4 ────> (2 min)
Agent 5 ────> (2 min)
Total: 2 minutes (5x speedup)
```
### Pattern Extraction Optimization
```python
# Quick mode (3 patterns/category): ~30 seconds
# Deep mode (5 patterns/category): ~60 seconds
# Optimization: Cache iteration scores
scores_cache = {}
def score_iteration(iteration, category):
cache_key = f"{iteration.id}_{category}"
if cache_key not in scores_cache:
scores_cache[cache_key] = compute_score(iteration, category)
return scores_cache[cache_key]
```
### I/O Optimization
```python
# Read all iterations once, keep in memory
iterations = [read_file(f) for f in iteration_files]
# Avoid repeated file I/O
for category in categories:
extract_patterns(iterations, category) # Uses in-memory data
```
## Extension Points
### Custom Pattern Categories
Add new pattern categories by:
1. Update `pattern_library_template.json`:
```json
{
"patterns": {
"structural": [...],
"content": [...],
"innovation": [...],
"quality": [...],
"performance": [...] // NEW CATEGORY
}
}
```
2. Update extraction logic in `extract-patterns.md`
3. Update validator to check new category
4. Update analysis to track new category adoption
### Custom Scoring Dimensions
Add new scoring dimensions:
```python
def score_iteration(iteration):
return {
"functionality": score_functionality(iteration),
"code_quality": score_code_quality(iteration),
"innovation": score_innovation(iteration),
"accessibility": score_accessibility(iteration), // NEW
"performance": score_performance(iteration), // NEW
}
```
### Custom Pattern Selection
Override default selection algorithm:
```python
def extract_patterns_custom(iterations, category, count=5):
# Custom logic: prefer patterns from recent iterations
recent_iterations = iterations[-10:]
return extract_patterns(recent_iterations, category, count)
```
## Security Considerations
### File System Access
- Validators only read pattern library (no writes)
- Sub-agents write only to designated output directory
- Pattern extraction reads only from output directory
- No execution of generated code during pattern extraction
### JSON Injection
- Pattern library validated with `jq` before use
- Malformed JSON fails gracefully
- No `eval()` or code execution from JSON
### Resource Limits
- Context budget prevents infinite loops
- Wave size capped (max 10 iterations per wave)
- Pattern library size capped (max 5 per category)
- File size limits on generated iterations (spec-dependent)
## Testing Architecture
### Unit Testing Pattern Extraction
```bash
# Create test iterations
mkdir test_iterations
echo "test content" > test_iterations/test_1.html
# Run extraction
/project:extract-patterns test_iterations test_patterns.json
# Validate output
./validators/check_patterns.sh test_patterns.json
```
### Integration Testing Full Loop
```bash
# Generate 10 iterations
/project:infinite-synthesis specs/example_spec.md test_output 10
# Verify outputs
ls test_output/*.html | wc -l # Should be 10
# Verify pattern library created
test -f pattern_library/patterns.json
# Verify pattern library valid
./validators/check_patterns.sh pattern_library/patterns.json
```
### Regression Testing
```bash
# Known-good pattern library
cp pattern_library/patterns.json pattern_library/baseline.json
# Generate with baseline
/project:infinite-synthesis specs/example_spec.md output_baseline 5 pattern_library/baseline.json
# Compare quality
/project:analyze-patterns pattern_library/baseline.json output_baseline
```
## Future Architecture Enhancements
### Planned Improvements
1. **Pattern Confidence Scores**
- Track success rate of each pattern
- Prioritize high-confidence patterns
- Deprecate low-confidence patterns
2. **Pattern Genealogy**
- Track which iteration created which pattern
- Visualize pattern evolution over waves
- Credit most influential iterations
3. **Cross-Spec Pattern Sharing**
- Export patterns for reuse across projects
- Import patterns from external sources
- Pattern library marketplace
4. **Adaptive Wave Sizing**
- Adjust wave size based on pattern stability
- Larger waves when patterns are stable
- Smaller waves during exploration phases
5. **Real-Time Quality Monitoring**
- Stream quality metrics during generation
- Early stopping if quality degrades
- Dynamic pattern injection
### Research Opportunities
1. **Optimal Pattern Count**: Is 3-5 truly optimal? A/B test different counts
2. **Pattern Decay**: Do patterns become less effective over time?
3. **Transfer Learning**: Can patterns from one domain help another?
4. **Human-in-the-Loop**: Manual pattern curation vs automatic extraction
5. **Pattern Combinations**: Identify synergistic pattern pairs
---
**Last Updated**: 2025-10-10
**Version**: 1.0
**Architecture Stability**: Stable (no breaking changes planned)
Reference in New Issue View Git Blame Copy Permalink