infinite-agents-public/infinite_variants/infinite_variant_4/CLAUDE.md

CLAUDE.md - Infinite Loop Variant 4: Quality Evaluation & Ranking System

This file provides guidance to Claude Code when working with the Quality Evaluation & Ranking System variant of the infinite agentic loop pattern.

Project Overview

This is Infinite Loop Variant 4, implementing automated quality evaluation and ranking for AI-generated iterations. The system uses the ReAct pattern (Reasoning + Acting + Observation) to evaluate, score, rank, and continuously improve iteration quality across multiple dimensions.

Key Concepts

ReAct Pattern Integration

Every operation in this system follows the ReAct cycle:

1. THOUGHT (Reasoning): Explicitly reason about quality, evaluation strategy, and improvement opportunities before acting
2. ACTION (Acting): Execute evaluations, generate content, score iterations with clear intent
3. OBSERVATION (Observing): Analyze results, identify patterns, extract insights to inform next cycle

Critical: Always document reasoning. Every evaluation, ranking, and report should show the thought process that led to conclusions.

Multi-Dimensional Quality

Quality is assessed across three equally important dimensions:

• Technical Quality (35%): Code, architecture, performance, robustness
• Creativity Score (35%): Originality, innovation, uniqueness, aesthetic
• Spec Compliance (30%): Requirements, naming, structure, standards

Critical: Never evaluate just one dimension. Quality is holistic. An iteration can be technically perfect but creatively bland, or wildly creative but technically flawed. Balance matters.

Quality-Driven Improvement

In infinite mode, quality assessment drives generation strategy:

• Early waves: Establish baseline, explore diversity
• Mid waves: Learn from top performers, address gaps
• Late waves: Push frontiers, optimize composite scores
• All waves: Monitor trends, adapt continuously

Critical: Don't just generate and evaluate. Learn from evaluations and adapt strategy accordingly. Let observations inform next actions.

Commands to Use

Primary Command: /project:infinite-quality

When to use: Generating iterations with quality evaluation and ranking

Syntax:

/project:infinite-quality <spec_path> <output_dir> <count|infinite> [config_path]

Key responsibilities when executing:

1. Initial Reasoning (THOUGHT):

   • Deeply understand spec quality criteria
   • Plan evaluation strategy
   • Design quality-driven creative directions
   • Consider what makes quality in this context

2. Generation (ACTION):

   • Launch sub-agents with quality targets
   • Provide spec + quality standards
   • Assign diverse creative directions
   • Generate with self-assessment

3. Evaluation (ACTION):

   • Score all iterations on all dimensions
   • Use evaluators from evaluators/ directory
   • Document evidence for all scores
   • Be fair, consistent, and thorough

4. Ranking & Reporting (OBSERVATION):

   • Rank by composite score
   • Identify patterns and trade-offs
   • Extract actionable insights
   • Generate comprehensive report

5. Strategy Adaptation (THOUGHT for next wave):

   • Learn from top performers
   • Address quality gaps
   • Adjust creative directions
   • Refine quality targets

Example execution flow:

User: /project:infinite-quality specs/example_spec.md output/ 10

You should:
1. Read and analyze specs/example_spec.md deeply
2. Read specs/quality_standards.md for evaluation criteria
3. Reason about quality goals for this spec (THOUGHT)
4. Launch 10 sub-agents with diverse creative directions (ACTION)
5. Evaluate all 10 iterations using evaluators/ logic (ACTION)
6. Rank iterations and generate quality report (OBSERVATION)
7. Present key findings and recommendations

Utility Command: /evaluate

When to use: Evaluating a single iteration on specific dimensions

Syntax:

/evaluate <dimension> <iteration_path> [spec_path]

Dimensions: technical, creativity, compliance, all

Key responsibilities:

1. Pre-Evaluation Reasoning (THOUGHT):

   • What does quality mean for this dimension?
   • What evidence should I look for?
   • How do I remain objective?

2. Evaluation (ACTION):

   • Read iteration completely
   • Load appropriate evaluator logic
   • Score each sub-dimension with evidence
   • Calculate total dimension score

3. Analysis (OBSERVATION):

   • Identify specific strengths
   • Identify specific weaknesses
   • Provide evidence for scores
   • Suggest improvements

Critical: Always provide specific evidence. Never say "code quality is good" without examples like "lines 45-67 demonstrate excellent input validation with clear error messages."

Utility Command: /rank

When to use: Ranking all iterations in a directory

Syntax:

/rank <output_dir> [dimension]

Key responsibilities:

1. Pre-Ranking Reasoning (THOUGHT):

   • What makes fair ranking?
   • What patterns to look for?
   • How to interpret rankings?

2. Ranking (ACTION):

   • Load all evaluations
   • Calculate composite scores
   • Sort and segment
   • Identify quality profiles

3. Pattern Analysis (OBSERVATION):

   • Quality clusters and outliers
   • Dimension trade-offs
   • Success/failure factors
   • Strategic recommendations

Critical: Rankings should reveal insights, not just order. Explain what separates top from bottom performers.

Utility Command: /quality-report

When to use: Generating comprehensive quality reports

Syntax:

/quality-report <output_dir> [wave_number]

Key responsibilities:

1. Pre-Report Reasoning (THOUGHT):

   • Purpose and audience
   • Most important insights
   • How to visualize quality

2. Report Generation (ACTION):

   • Aggregate all evaluation data
   • Calculate comprehensive statistics
   • Generate text visualizations
   • Identify patterns and insights

3. Strategic Recommendations (OBSERVATION):

   • Actionable next steps
   • Creative direction suggestions
   • Quality targets for next wave
   • System improvements

Critical: Reports must be actionable. Every insight should lead to a concrete recommendation.

Evaluation Guidelines

Technical Quality Evaluation

Focus on:

• Code Quality: Readability, comments, naming, DRY
• Architecture: Modularity, separation, reusability, scalability
• Performance: Render speed, animation fps, algorithms, DOM ops
• Robustness: Validation, error handling, edge cases, compatibility

Scoring approach:

• Look for concrete evidence in code
• Compare against standards in evaluators/technical_quality.md
• Score each sub-dimension 0-25 points
• Document specific examples
• Total: 0-100 points

Example evidence:

• Good: "Lines 120-145: Efficient caching mechanism reduces redundant calculations"
• Bad: "Performance is good"

Creativity Score Evaluation

Focus on:

• Originality: Novel concepts, fresh perspectives, unexpected approaches
• Innovation: Creative solutions, clever techniques, boundary-pushing
• Uniqueness: Differentiation from others, distinctive identity, memorability
• Aesthetic: Visual appeal, color harmony, typography, polish

Scoring approach:

• Recognize creativity is partially subjective
• Look for objective indicators of novelty
• Compare against standards in evaluators/creativity_score.md
• Reward creative risk-taking
• Total: 0-100 points

Example evidence:

• Good: "Novel data-as-music-notation concept, first iteration to use audio sonification"
• Bad: "This is creative"

Spec Compliance Evaluation

Focus on:

• Requirements Met: Functional, technical, design requirements (40 points)
• Naming Conventions: Pattern adherence, quality (20 points)
• Structure Adherence: File structure, code organization (20 points)
• Quality Standards: Baselines met (20 points)

Scoring approach:

• Treat spec as checklist
• Binary or proportional scoring per requirement
• Compare against standards in evaluators/spec_compliance.md
• Be objective and evidence-based
• Total: 0-100 points

Example evidence:

• Good: "Spec requires 20+ data points, iteration has 50 points ✓ [4/4 points]"
• Bad: "Meets requirements"

Scoring Calibration

Use these reference points to ensure consistent scoring:

90-100 (Exceptional): Excellence in all sub-dimensions, exemplary work 80-89 (Excellent): Strong across all sub-dimensions, minor improvements possible 70-79 (Good): Solid in most sub-dimensions, some areas need work 60-69 (Adequate): Meets basic requirements, notable weaknesses 50-59 (Needs Improvement): Below expectations, significant issues Below 50 (Insufficient): Major deficiencies, fails basic criteria

Critical: Most iterations should fall in 60-85 range. Scores of 90+ should be rare and truly exceptional. Scores below 60 indicate serious problems.

Quality Report Best Practices

When generating quality reports:

1. Start with Executive Summary: 3 key insights, 1 priority recommendation
2. Provide Statistics: Mean, median, std dev, min, max for all dimensions
3. Visualize Distribution: Text-based histograms and charts
4. Identify Patterns: What makes top iterations succeed? What causes low scores?
5. Analyze Trade-offs: Which dimensions compete? Which synergize?
6. Give Strategic Recommendations: Specific, actionable, prioritized
7. Self-Assess Report Quality: Is this useful? Honest? Comprehensive?

Critical: Every report should drive improvement. If it doesn't lead to actionable insights, it's not a good report.

Infinite Mode Strategy

When running in infinite mode (count: infinite):

Wave 1 (Foundation)

• Generate 6-8 iterations with diverse creative directions
• Establish baseline quality metrics
• Identify initial strengths and weaknesses
• Generate wave 1 report

Wave 2+ (Progressive Improvement)

THOUGHT Phase:

• What made wave 1 top performers successful?
• What quality dimensions need improvement?
• What creative directions are underexplored?
• How can we push quality higher?

ACTION Phase:

• Generate 6-8 new iterations
• Incorporate lessons from previous waves
• Target quality gaps
• Increase challenge in strong areas

OBSERVATION Phase:

• Evaluate new iterations
• Update overall rankings
• Generate wave-specific report
• Compare to previous waves

Adaptation:

• Quality improving? Continue strategy
• Quality stagnating? Adjust approach
• Quality declining? Investigate and correct

Critical: Don't just repeat same strategy. Each wave should learn from previous waves. Show progressive improvement.

Configuration Customization

Users can customize scoring through config/scoring_weights.json:

Default weights:

• Technical: 35%, Creativity: 35%, Compliance: 30%

Alternative profiles:

• technical_focus: 50/25/25 - For production code
• creative_focus: 25/50/25 - For exploratory projects
• compliance_focus: 30/25/45 - For standardization
• innovation_priority: 20/60/20 - For research

When using custom config:

1. Load and validate config
2. Apply weights to composite score calculation
3. Document which config is being used
4. Note how it affects scoring

Common Pitfalls to Avoid

1. Evaluation without Reasoning: Don't just score - explain why
2. Inconsistent Scoring: Apply same criteria to all iterations
3. Vague Feedback: Provide specific evidence and examples
4. Ignoring Trade-offs: Recognize when dimensions compete
5. Not Learning from Results: Use observations to inform next actions
6. Artificial Precision: Don't pretend scores are more accurate than they are
7. Forgetting Balance: All three dimensions matter

Success Criteria

A successful execution of this system demonstrates:

1. Meaningful Differentiation: Scores clearly separate quality levels
2. Evidence-Based Scoring: Every score backed by specific examples
3. Actionable Insights: Reports lead to concrete improvements
4. Visible Learning: Quality improves over waves in infinite mode
5. Transparent Reasoning: ReAct pattern evident throughout
6. Fair Consistency: Same criteria applied to all iterations

File Organization

When generating iterations:

{output_dir}/
├── iteration_001.html
├── iteration_002.html
├── ...
└── quality_reports/
    ├── evaluations/
    │   ├── iteration_001_evaluation.json
    │   ├── iteration_002_evaluation.json
    │   └── ...
    ├── rankings/
    │   ├── ranking_report.md
    │   └── ranking_data.json
    └── reports/
        ├── wave_1_report.md
        ├── wave_2_report.md
        └── ...

Critical: Keep quality data organized. Store evaluations as JSON for machine readability, reports as Markdown for human readability.

Integration with Main Infinite Loop Project

This variant builds on the original infinite loop pattern with quality-focused enhancements:

Shared concepts:

• Multi-agent parallel orchestration
• Specification-driven generation
• Wave-based iteration (infinite mode)
• Context management

New additions:

• ReAct reasoning pattern
• Multi-dimensional quality evaluation
• Automated ranking and reporting
• Quality-driven strategy adaptation

Critical: This is not a replacement for the original pattern - it's an enhancement focused on quality assessment and continuous improvement.

Examples

Example 1: Small Batch with Quality Focus

User: /project:infinite-quality specs/example_spec.md output/ 5

You should:
1. Analyze spec quality criteria
2. Reason about quality goals (THOUGHT)
3. Generate 5 diverse iterations (ACTION)
4. Evaluate all on all dimensions (ACTION)
5. Rank and report (OBSERVATION)
6. Provide top 3 insights

Expected output:
- 5 HTML files in output/
- 5 evaluation JSON files in output/quality_reports/evaluations/
- 1 ranking report in output/quality_reports/rankings/
- 1 quality report in output/quality_reports/reports/
- Summary with key findings and recommendations

Example 2: Infinite Mode with Learning

User: /project:infinite-quality specs/example_spec.md output/ infinite

You should:
Wave 1:
- Generate 6-8 iterations
- Evaluate and rank
- Report baseline quality

Wave 2:
- THOUGHT: Learn from wave 1 top performers
- ACTION: Generate 6-8 new iterations with lessons applied
- OBSERVATION: Evaluate, rank, report improvements

Wave 3+:
- Continue THOUGHT → ACTION → OBSERVATION cycle
- Show progressive quality improvement
- Adapt strategy based on observations
- Continue until context limits

Expected pattern:
- Quality scores increase over waves
- Strategy evolves based on observations
- Reports show learning and adaptation

When to Ask for Clarification

Ask user if:

• Spec lacks quality criteria (offer to use defaults)
• Custom config has invalid weights (offer to fix)
• Unclear whether to prioritize technical vs creative
• Infinite mode strategy needs direction
• Evaluation criteria should be adjusted

Don't ask about:

• How to score (use evaluators/ logic)
• Report format (use templates/ structure)
• Ranking methodology (use rank.md process)
• Standard evaluation process (documented in commands)

Version & Maintenance

Current Version: 1.0 Created: 2025-10-10 Pattern: Infinite Agentic Loop + ReAct Reasoning Dependencies: Claude Code custom commands, WebFetch for ReAct pattern research

Future considerations:

• Automated testing integration
• Visual quality report generation
• Meta-learning on evaluation criteria
• User feedback integration

---

Remember: Quality evaluation is not about being harsh or lenient - it's about being fair, consistent, and helpful. Use the ReAct pattern to reason thoughtfully, act systematically, and observe honestly. Let quality assessment drive continuous improvement.