infinite-agents-public/infinite_variants/infinite_variant_5/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code when working with the Configuration-Driven Infinite Loop Orchestration variant.

## Project Overview

This is **Variant 5** of the Infinite Agents project, implementing a **configuration-driven orchestration system** with **chain prompting** patterns.

Unlike previous variants with hardcoded parameters, this variant externalizes all orchestration settings to JSON configuration files, enabling flexible, reproducible, and production-ready infinite loop execution.

## Core Innovation

### Configuration-Driven Architecture
All orchestration parameters are externalized to configuration files:
- Defaults in `.claude/config/defaults.json`
- Profiles in `.claude/config/profiles/{profile}.json`
- Custom configurations via file paths or inline JSON
- Hierarchical merging: defaults → profile → custom → runtime overrides

### Chain Prompting Implementation
Implements chain prompting pattern from Anthropic documentation:
- Workflow decomposed into 7 focused stages
- State passed between stages via XML tags
- Each stage has single-task goal for maximum attention
- Self-correction loops for quality enforcement
- Complete traceability through workflow audit trail

## Key Commands

### Primary Command: /project:infinite-config

Main orchestration command implementing full chain prompting workflow.

**Syntax:**
```bash
/project:infinite-config <spec_path> <output_dir> <count> [profile] [config_overrides]
```

**Arguments:**
- `spec_path`: Path to specification file
- `output_dir`: Output directory (relative to project root)
- `count`: Number of iterations or "infinite"
- `profile` (optional): Configuration profile (development|production|research)
- `config_overrides` (optional): Path to custom config JSON or inline JSON

**Examples:**
```bash
# Use defaults
/project:infinite-config specs/example_spec.md output 5

# Use development profile
/project:infinite-config specs/example_spec.md output_dev 3 development

# Use production profile
/project:infinite-config specs/example_spec.md output_prod 20 production

# Use custom config file
/project:infinite-config specs/example_spec.md output 10 custom examples/custom_config.json

# Use inline overrides
/project:infinite-config specs/example_spec.md output 5 development '{"orchestration":{"max_parallel_agents":8}}'

# Infinite mode
/project:infinite-config specs/example_spec.md infinite_output infinite production
```

### Utility Commands

**Configuration Validation:**
```bash
/project:validate-config [config_path] [profile]
```

Validates configuration files with:
- Schema validation (types, required fields, constraints)
- Semantic validation (logical consistency, value reasonableness)
- Cross-field validation (relationship checks, compatibility)

**Interactive Configuration:**
```bash
/project:configure [action] [profile] [output_path]
```

Actions:
- `create` - Create new custom configuration interactively
- `edit` - Edit existing configuration
- `compare` - Compare two configurations
- `optimize` - Optimize for use case (speed, quality, scale, development, research)
- `merge` - Merge multiple configurations

## Configuration Profiles

### Development Profile
**File:** `.claude/config/profiles/development.json`

Optimized for:
- Quick iteration and testing
- Debugging and learning
- Quality over speed

Settings:
- Small batches (3 iterations)
- 2 parallel agents
- Verbose logging (debug level)
- Review stage enabled
- Lower uniqueness threshold (0.7)
- Max 10 iterations
- Web enhancement enabled with caching

**Use when:**
- Testing new specifications
- Debugging issues
- Learning the system
- Developing new features

### Production Profile
**File:** `.claude/config/profiles/production.json`

Optimized for:
- Scale and throughput
- Efficiency and performance
- Minimal overhead

Settings:
- Large batches (10 iterations)
- 5 parallel agents
- Minimal logging (warn level)
- Review stage disabled
- High uniqueness threshold (0.9)
- Max 1000 iterations
- Web enhancement enabled, no caching
- Progressive sophistication enabled

**Use when:**
- Running large batches
- Production deployments
- Performance is critical
- Quality thresholds are well-tested

### Research Profile
**File:** `.claude/config/profiles/research.json`

Optimized for:
- Maximum quality
- Exploration and experimentation
- Comprehensive analysis

Settings:
- Medium batches (5 iterations)
- 3 parallel agents
- Extensive logging (debug level, verbose)
- Review stage enabled
- Very high uniqueness threshold (0.95)
- Max 50 iterations
- Extensive web priming (8 URLs)
- Cross-iteration learning enabled
- Additional chain prompting stages (11 total)

**Use when:**
- Exploring new domains
- Maximum quality needed
- Understanding system behavior
- Experimenting with techniques

## Chain Prompting Workflow

The system implements a 7-stage chain prompting workflow (configurable):

### Stage 1: Load Configuration
**Single-Task Goal:** Load and parse all configuration sources

Loads configurations in hierarchical order:
1. defaults.json
2. profile.json (if specified)
3. custom config file (if provided)
4. inline JSON overrides (if provided)

### Stage 2: Validate Configuration
**Single-Task Goal:** Validate all configurations against schema

Validates:
- Schema compliance (types, required fields, constraints)
- Value ranges and enum values
- String patterns

### Stage 3: Merge Configuration
**Single-Task Goal:** Merge validated configurations hierarchically

Deep merges all configurations, later values override earlier ones.

### Stage 4: Analyze Specification
**Single-Task Goal:** Deeply analyze the specification file

Extracts:
- Content requirements
- Quality standards
- Naming conventions
- Technical requirements

### Stage 5: Plan Execution
**Single-Task Goal:** Create detailed execution plan

Plans:
- Execution mode (batch vs infinite)
- Wave structure and agent assignments
- Web enhancement strategy
- Resource allocation

### Stage 6: Execute Generation
**Single-Task Goal:** Execute parallel agent generation

Executes:
- Initial web priming (if enabled)
- Parallel agent waves
- Error handling and retries
- Limit checking

### Stage 7: Validate Output (Optional)
**Single-Task Goal:** Validate generated outputs

Validates:
- Spec compliance
- Uniqueness thresholds
- File size limits
- Code quality

Enables self-correction loops if validation fails.

## Configuration System

### Configuration Sections

**orchestration** - Parallel execution settings
- `max_parallel_agents` (1-10): Simultaneous sub-agents
- `batch_size` (1-50): Iterations per batch
- `infinite_mode_wave_size` (1-20): Iterations per wave
- `context_budget_per_agent` (10000-200000): Token budget
- `agent_timeout_ms` (30000-600000): Execution timeout
- `enable_progressive_sophistication` (boolean): Increase complexity over time

**generation** - Output file settings
- `output_directory`: Save location
- `naming_pattern`: File naming with placeholders
- `file_format`: html|markdown|json|text
- `include_metadata` (boolean): Embed metadata
- `metadata_format`: html_comment|yaml_frontmatter|json_header

**quality** - Quality control
- `min_uniqueness_threshold` (0.0-1.0): Required uniqueness
- `enable_validation` (boolean): Validate outputs
- `enable_review_stage` (boolean): Add review step
- `max_retry_attempts` (0-5): Retry failures
- `require_spec_compliance_check` (boolean): Validate spec compliance

**web_enhancement** - Web learning
- `enabled` (boolean): Enable web-enhanced generation
- `initial_priming_urls` (0-10): Initial knowledge priming
- `urls_per_iteration` (0-5): URLs per iteration
- `progressive_difficulty` (boolean): Progressive URL difficulty
- `enable_web_search_fallback` (boolean): Search when exhausted
- `cache_web_content` (boolean): Cache fetched content
- `web_fetch_timeout_ms` (5000-60000): Fetch timeout

**logging** - Logging configuration
- `level`: debug|info|warn|error
- `log_agent_outputs` (boolean): Log sub-agent outputs
- `log_web_fetches` (boolean): Log web operations
- `log_config_loading` (boolean): Log config loading
- `verbose` (boolean): Maximum detail

**chain_prompting** - Chain prompting settings
- `enabled` (boolean): Enable chain prompting
- `stages` (array): Workflow stages to execute
- `enable_self_correction` (boolean): Enable self-correction
- `pass_state_via_xml` (boolean): Use XML for state

**features** - Feature flags
- `enable_url_strategy` (boolean): URL strategy files
- `enable_theme_evolution` (boolean): Evolve themes
- `enable_cross_iteration_learning` (boolean): Learn from previous
- `enable_automatic_indexing` (boolean): Auto-generate indexes

**limits** - Safety limits
- `max_iterations`: Maximum iterations
- `max_file_size_kb`: Maximum file size
- `max_total_output_mb`: Maximum total output
- `warn_at_iteration`: Warning threshold

### Configuration Hierarchy

Configurations merge in order (later overrides earlier):
1. `.claude/config/defaults.json` (base)
2. `.claude/config/profiles/{profile}.json` (if specified)
3. Custom config file (if path provided)
4. Inline JSON overrides (if provided)

Example:
```bash
# This uses: defaults ← production ← custom_config.json ← inline override
/project:infinite-config specs/example_spec.md output 20 production examples/custom_config.json '{"logging":{"verbose":true}}'
```

## Typical Workflows

### Workflow 1: Quick Test (Development)
```bash
# Validate development profile
/project:validate-config development

# Run 3 iterations for testing
/project:infinite-config specs/example_spec.md test_output 3 development
```

### Workflow 2: Production Batch
```bash
# Validate production profile
/project:validate-config production

# Run 50 iterations
/project:infinite-config specs/example_spec.md prod_output 50 production
```

### Workflow 3: Custom Configuration
```bash
# Create custom configuration interactively
/project:configure create production my_custom.json

# Validate it
/project:validate-config my_custom.json

# Run with custom config
/project:infinite-config specs/example_spec.md output 20 custom my_custom.json
```

### Workflow 4: Optimization
```bash
# Create speed-optimized configuration
/project:configure optimize speed

# Or quality-optimized
/project:configure optimize quality

# Compare configurations
/project:configure compare development production
```

### Workflow 5: Research Exploration
```bash
# Use research profile for maximum quality
/project:infinite-config specs/example_spec.md research_output 20 research

# Check results, then continue
/project:infinite-config specs/example_spec.md research_output 30 research
```

## Best Practices

### For Claude Code Agent

1. **Always validate configurations before running**
   - Use `/project:validate-config` to check
   - Fix errors before executing

2. **Choose appropriate profile for task**
   - Development: testing, debugging, learning
   - Production: scale, efficiency
   - Research: quality, exploration

3. **Use inline overrides for one-off changes**
   - Don't create new config file for single parameter change
   - Use inline JSON for temporary overrides

4. **Monitor resource usage**
   - Check context budget consumption
   - Monitor output directory size
   - Watch for limit warnings

5. **Start small, scale up**
   - Test with small batches first
   - Validate quality before large runs
   - Use development profile for initial tests

6. **Leverage chain prompting stages**
   - Each stage focuses on single task
   - State passes via XML tags
   - Self-correction enabled for quality

### For Specifications

1. **Reference configuration in spec**
   - Specs should note configurable parameters
   - Guide users to appropriate settings

2. **Set reasonable defaults**
   - Specs work with default configuration
   - Optimize for common use cases

3. **Document quality thresholds**
   - What uniqueness threshold is appropriate?
   - Which validation checks are needed?

## File Structure

```
infinite_variant_5/
├── .claude/
│   ├── commands/           # Slash commands
│   │   ├── infinite-config.md
│   │   ├── validate-config.md
│   │   └── configure.md
│   ├── config/             # Configuration system
│   │   ├── defaults.json
│   │   ├── schema.json
│   │   └── profiles/
│   │       ├── development.json
│   │       ├── production.json
│   │       └── research.json
│   └── settings.json
├── specs/                  # Specifications
│   └── example_spec.md
├── examples/               # Example configs
│   └── custom_config.json
├── docs/                   # Documentation
│   └── configuration_guide.md
├── README.md
└── CLAUDE.md              # This file
```

## Troubleshooting

### Configuration Errors
**Issue:** Configuration validation fails

**Solution:**
1. Run `/project:validate-config` to see specific errors
2. Check schema.json for valid values
3. Compare to working profile configurations
4. Use `/project:configure edit` to fix interactively

### Execution Failures
**Issue:** Generation fails during execution

**Solution:**
1. Check configured timeouts (agent_timeout_ms)
2. Verify output directory is writable
3. Check resource limits (max_iterations, max_total_output_mb)
4. Enable verbose logging to diagnose
5. Try smaller batch_size

### Uniqueness Issues
**Issue:** Iterations too similar

**Solution:**
1. Increase `min_uniqueness_threshold`
2. Enable `enable_theme_evolution`
3. Check specification diversity requirements
4. Review existing iterations for patterns

### Performance Issues
**Issue:** Execution too slow

**Solution:**
1. Increase `max_parallel_agents` (up to 10)
2. Disable `enable_review_stage`
3. Reduce logging level to `warn` or `error`
4. Disable web enhancement if not needed
5. Use production profile as baseline

### Web Enhancement Issues
**Issue:** Web fetches failing or slow

**Solution:**
1. Check `web_fetch_timeout_ms` is sufficient
2. Enable `enable_web_search_fallback`
3. Reduce `urls_per_iteration`
4. Enable `cache_web_content` to reuse fetches

## Advanced Usage

### Custom Chain Prompting Stages

Modify `chain_prompting.stages` array to customize workflow:

```json
{
  "chain_prompting": {
    "stages": [
      "load_config",
      "validate_config",
      "merge_config",
      "analyze_spec",
      "research_existing",      // Custom stage
      "plan_execution",
      "execute_generation",
      "review_output",          // Optional stage
      "refine_output",          // Custom stage
      "validate_output",
      "analyze_results"         // Custom stage
    ]
  }
}
```

### Progressive Sophistication

When `enable_progressive_sophistication` is true, infinite mode increases complexity:

- Wave 1: Basic implementations
- Wave 2: Intermediate complexity
- Wave 3: Advanced techniques
- Wave 4+: Expert level

Configure wave size with `infinite_mode_wave_size`.

### Cross-Iteration Learning

Enable with `features.enable_cross_iteration_learning`:
- Later iterations learn from earlier ones
- Requires additional context budget
- Improves quality but increases execution time

### Automatic Indexing

Enable with `features.enable_automatic_indexing`:
- Generates index.html after each wave
- Lists all generated iterations
- Provides navigation and preview

## Integration with Other Variants

This configuration system can be adapted to other variants:

1. **Variant 1 (Original)**: Add configuration layer
2. **Variant 2 (Web-Enhanced)**: Already supports web enhancement config
3. **Variant 3 (State-Based)**: Add state management config section
4. **Variant 4 (Specialized Agents)**: Add agent role configurations

## Further Reading

- `README.md` - Overview and quick start
- `docs/configuration_guide.md` - Complete configuration reference
- `.claude/commands/infinite-config.md` - Full command documentation
- `.claude/commands/validate-config.md` - Validation documentation
- `.claude/commands/configure.md` - Interactive utility documentation
- `.claude/config/schema.json` - Complete schema reference
- `specs/example_spec.md` - Example specification

## Notes

- Configuration files use JSON format with JSON Schema validation
- All file paths should be relative to project root
- Inline JSON overrides must be valid JSON strings
- Chain prompting stages execute sequentially
- Self-correction loops respect `max_retry_attempts`
- Context budget managed across all stages and agents