# Analyze Pattern Library Effectiveness

Evaluate how well the pattern library is improving iteration quality.

## Usage

```bash
/project:analyze-patterns <pattern_library_path> <iterations_dir>
```

## Arguments

1. `pattern_library_path` - Path to pattern library JSON file
2. `iterations_dir` - Directory containing iterations to analyze

## Examples

```bash
# Analyze pattern effectiveness
/project:analyze-patterns pattern_library/patterns.json output

# Generate detailed metrics report
/project:analyze-patterns pattern_library/patterns.json output
```

## How It Works

This command measures the effectiveness of pattern-guided generation:

1. **Load Pattern Library**: Read current patterns and metadata
2. **Iteration Analysis**: Examine all iterations for pattern adoption
3. **Quality Comparison**: Compare pre-pattern vs post-pattern iterations
4. **Pattern Attribution**: Identify which patterns are most adopted
5. **Effectiveness Report**: Generate metrics showing pattern impact

## Implementation Steps

### Step 1: Load Pattern Library

```bash
# Read pattern library
Read pattern_library_path

# Parse JSON and extract:
- Total patterns per category
- Pattern characteristics
- Example files
- Success metrics
```

### Step 2: Categorize Iterations

```bash
# List all iterations chronologically
Bash: ls -lt iterations_dir

# Determine which iterations were generated before/after pattern library:
- Pre-pattern iterations: Generated before library creation
- Post-pattern iterations: Generated with pattern guidance
```

### Step 3: Pattern Adoption Analysis

For each post-pattern iteration:

```markdown
Analyze file content to detect pattern usage:

Structural patterns:
- Check for modular architecture
- Verify naming conventions
- Identify organizational patterns
- Match against library examples

Content patterns:
- Evaluate documentation quality
- Check comment patterns
- Assess clarity metrics
- Compare to library standards

Innovation patterns:
- Look for creative techniques from library
- Identify novel applications of patterns
- Detect pattern combinations

Quality patterns:
- Check for validation logic
- Identify error handling approaches
- Verify testing patterns
- Measure robustness
```

Calculate **Pattern Adoption Rate**:

```
Adoption Rate = (Iterations using 1+ patterns) / (Total post-pattern iterations)
```

### Step 4: Quality Comparison

Compare iterations before and after pattern library:

```markdown
Pre-Pattern Iterations:
- Average quality score: {score}
- Structural consistency: {variance}
- Innovation diversity: {count}
- Common issues: {list}

Post-Pattern Iterations:
- Average quality score: {score}
- Structural consistency: {variance}
- Innovation diversity: {count}
- Common issues: {list}

Improvement Metrics:
- Quality increase: {percent}%
- Consistency improvement: {percent}%
- Innovation increase: {count}
- Issue reduction: {percent}%
```

### Step 5: Pattern Impact Ranking

Rank patterns by their impact:

```json
{
  "most_adopted_patterns": [
    {
      "pattern_name": "Modular Three-Layer Architecture",
      "category": "structural",
      "adoption_count": 8,
      "adoption_rate": "80%",
      "avg_quality_improvement": "+15%"
    },
    {
      "pattern_name": "Progressive Disclosure Documentation",
      "category": "content",
      "adoption_count": 6,
      "adoption_rate": "60%",
      "avg_quality_improvement": "+12%"
    }
  ],
  "least_adopted_patterns": [
    {
      "pattern_name": "Self-Validating Data Pipeline",
      "category": "innovation",
      "adoption_count": 2,
      "adoption_rate": "20%",
      "possible_reasons": ["Too complex", "Not applicable to all specs"]
    }
  ]
}
```

### Step 6: Pattern Evolution Analysis

Track how patterns have evolved across versions:

```markdown
Pattern Library Version History:
- v1.0 (Wave 1): 12 patterns extracted
- v1.1 (Wave 2): 13 patterns (1 new structural pattern)
- v1.2 (Wave 3): 14 patterns (1 new innovation pattern)

Pattern Turnover:
- Patterns removed: 2 (replaced by better examples)
- Patterns added: 4
- Patterns refined: 3
- Stable patterns: 10
```

### Step 7: Multi-Shot Effectiveness

Evaluate how well patterns serve as examples (multi-shot prompting):

```markdown
Multi-Shot Prompting Metrics:

Example Clarity:
- Patterns with clear code snippets: {count}/{total}
- Patterns with measurable success metrics: {count}/{total}
- Patterns with diverse examples: {count}/{total}

Example Impact:
- Iterations citing pattern examples: {count}
- Average patterns used per iteration: {number}
- Pattern combination frequency: {percent}%

Example Quality:
- Patterns from top 20% iterations: {percent}%
- Pattern diversity score: {score}/10
- Pattern transferability: {score}/10
```

### Step 8: Generate Effectiveness Report

Create comprehensive analysis report:

```markdown
# Pattern Library Effectiveness Report

**Generated**: 2025-10-10T15:00:00Z
**Pattern Library**: pattern_library/patterns.json (v1.2)
**Iterations Analyzed**: 20

## Executive Summary

The pattern library has improved iteration quality by **{percent}%** and increased structural consistency by **{percent}%**. Pattern adoption rate is **{percent}%**, indicating strong effectiveness.

## Key Findings

### Pattern Adoption
- **Total Iterations**: 20 (10 pre-pattern, 10 post-pattern)
- **Adoption Rate**: 80% (8/10 post-pattern iterations use patterns)
- **Avg Patterns per Iteration**: 3.2
- **Most Common Pattern**: Modular Three-Layer Architecture (80% adoption)

### Quality Improvement
- **Pre-Pattern Quality**: 7.2/10 average
- **Post-Pattern Quality**: 8.8/10 average
- **Improvement**: +22%
- **Consistency**: Variance reduced from 1.8 to 0.6

### Pattern Impact Rankings

#### Most Effective Patterns
1. **Modular Three-Layer Architecture** (Structural)
   - Adoption: 80%
   - Quality Impact: +15%
   - Why: Clear structure, easy to replicate

2. **Progressive Disclosure Documentation** (Content)
   - Adoption: 60%
   - Quality Impact: +12%
   - Why: Improves readability, scalable approach

3. **Guard Clause Pattern with Fallbacks** (Quality)
   - Adoption: 50%
   - Quality Impact: +18%
   - Why: Prevents errors, improves robustness

#### Least Adopted Patterns
1. **Self-Validating Data Pipeline** (Innovation)
   - Adoption: 20%
   - Reason: Complex, not applicable to all specs

2. **{Pattern Name}** ({Category})
   - Adoption: {percent}%
   - Reason: {explanation}

### Pattern Evolution
- **Library Versions**: 1.0 → 1.2 (3 waves)
- **Patterns Added**: 4
- **Patterns Removed**: 2
- **Stable Core**: 10 patterns remain consistent

### Innovation Impact
- **Pre-Pattern**: 12 unique innovations
- **Post-Pattern**: 18 unique innovations
- **Change**: +50% increase
- **Observation**: Patterns provide foundation, enabling more innovation

## Multi-Shot Prompting Analysis

### Example Quality
- ✓ All patterns include code snippets
- ✓ 95% have measurable success metrics
- ✓ Diverse examples (3-5 per category)

### Example Effectiveness
- **Pattern Citation Rate**: 75%
- **Average Patterns per Iteration**: 3.2
- **Pattern Combination**: 40% of iterations combine 2+ patterns

### Example Consistency
- **Uniform Structure**: All patterns follow JSON schema
- **Clear Success Metrics**: 95% of patterns
- **Transferability**: 85% applicable across different specs

## Recommendations

### High-Priority Actions
1. **Promote Top Patterns**: Feature most effective patterns prominently
2. **Refine Low-Adoption Patterns**: Simplify or provide better examples
3. **Document Pattern Combinations**: Show successful pattern pairings
4. **Expand Success Metrics**: Add quantitative measurements

### Pattern Library Improvements
1. Add "Pattern Combination" category for synergistic patterns
2. Include anti-patterns (what NOT to do) for contrast
3. Provide minimal vs maximal examples of each pattern
4. Create pattern decision tree for easier selection

### Future Analysis
1. Track pattern effectiveness over longer time periods
2. A/B test pattern-guided vs non-pattern iterations
3. Measure context efficiency (patterns reduce context needs?)
4. Survey agent "preferences" for certain patterns

## Visualizations

### Quality Score Distribution
```
Pre-Pattern:  [====    ] 7.2/10 avg (variance: 1.8)
Post-Pattern: [========] 8.8/10 avg (variance: 0.6)
```

### Pattern Adoption Over Time
```
Wave 1:  [          ] 0% (no patterns yet)
Wave 2:  [======    ] 60% adoption
Wave 3:  [========  ] 80% adoption
Wave 4:  [========= ] 90% adoption (projected)
```

### Top Patterns by Category
```
Structural:  Modular Three-Layer [========] 80%
Content:     Progressive Disclosure [======] 60%
Innovation:  Novel Data Binding [====] 40%
Quality:     Guard Clause [=====] 50%
```

## Conclusion

The pattern library demonstrates strong effectiveness as a multi-shot prompting mechanism. Pattern adoption rate of **{percent}%** and quality improvement of **{percent}%** validate the approach. Continued refinement and expansion of the library will further enhance iteration quality and consistency.

**Next Steps**: Continue pattern extraction after each wave, focusing on emerging patterns and successful combinations.

---

**Pattern Library Location**: {pattern_library_path}
**Report Generated**: 2025-10-10T15:00:00Z
```

## Metrics Tracked

This command calculates and reports:

1. **Adoption Metrics**
   - Pattern adoption rate
   - Patterns per iteration
   - Most/least adopted patterns

2. **Quality Metrics**
   - Pre/post quality comparison
   - Consistency improvement
   - Error rate reduction

3. **Innovation Metrics**
   - Unique innovations count
   - Pattern combinations
   - Novel pattern applications

4. **Evolution Metrics**
   - Library version progression
   - Pattern turnover rate
   - Stable vs emerging patterns

5. **Multi-Shot Effectiveness**
   - Example clarity scores
   - Example impact measures
   - Example quality validation

## Validation

The analysis ensures:

```markdown
- Sufficient data: At least 5 iterations analyzed
- Version tracking: Pattern library versions are sequential
- Quality scoring: Consistent methodology applied
- Attribution accuracy: Patterns correctly identified in iterations
- Statistical validity: Comparisons are meaningful
```

## Notes

- Analysis should be run after each wave to track progression
- Metrics help identify which patterns to keep/remove/refine
- Quality improvements validate the pattern synthesis approach
- Low adoption patterns may need better examples or documentation
- This analysis informs pattern library curation decisions

## Related Commands

- `/project:infinite-synthesis` - Main loop generating iterations
- `/project:extract-patterns` - Extract patterns from iterations