561 lines
16 KiB
Markdown
561 lines
16 KiB
Markdown
# Web-Enhanced Infinite Agentic Loop Guide
|
|
|
|
## Overview
|
|
|
|
The **Web-Enhanced Infinite Agentic Loop** extends the original infinite loop pattern with progressive web-based knowledge acquisition. Each iteration fetches and learns from web resources, creating a self-improving system that gets smarter with each generation.
|
|
|
|
## Core Innovation
|
|
|
|
**Traditional Loop:**
|
|
```
|
|
Spec → Generate → Iterate → Generate → Iterate...
|
|
```
|
|
|
|
**Web-Enhanced Loop:**
|
|
```
|
|
Spec → Web Prime → [Fetch URL → Learn → Generate]* → Iterate...
|
|
↑_______________|
|
|
```
|
|
|
|
Each iteration:
|
|
1. Fetches a NEW web resource (URL)
|
|
2. Learns specific techniques from that resource
|
|
3. Applies learnings to create improved output
|
|
4. Documents what was learned and how it was applied
|
|
|
|
## System Components
|
|
|
|
### 1. Specification File (`specs/*.md`)
|
|
Defines what to generate and how web research should enhance it.
|
|
|
|
**Example:** `specs/d3_visualization_progressive.md`
|
|
- Output format and structure
|
|
- Quality standards
|
|
- Progressive enhancement dimensions
|
|
- Web integration strategy
|
|
- Success criteria
|
|
|
|
### 2. URL Strategy File (`specs/*_url_strategy.json`)
|
|
Curated knowledge pathway with URLs organized by difficulty.
|
|
|
|
**Structure:**
|
|
```json
|
|
{
|
|
"priming_urls": [...], // Initial research URLs
|
|
"foundation_urls": [...], // Iterations 1-5
|
|
"intermediate_urls": [...], // Iterations 6-12
|
|
"advanced_urls": [...], // Iterations 13-20
|
|
"expert_urls": [...], // Iterations 21+
|
|
"web_search_templates": [...] // Dynamic search fallbacks
|
|
}
|
|
```
|
|
|
|
### 3. Command: `/project:infinite-web`
|
|
Orchestrates the web-enhanced generation process.
|
|
|
|
**Location:** `.claude/commands/infinite-web.md`
|
|
|
|
## Usage
|
|
|
|
### Basic Command Structure
|
|
|
|
```bash
|
|
/project:infinite-web <spec_file> <output_dir> <count> [url_strategy_file]
|
|
```
|
|
|
|
**Parameters:**
|
|
- `spec_file`: Path to specification markdown
|
|
- `output_dir`: Where to save generated files
|
|
- `count`: Number of iterations (1-N or "infinite")
|
|
- `url_strategy_file`: (Optional) Path to URL strategy JSON
|
|
|
|
### Usage Examples
|
|
|
|
#### 1. Single D3 Visualization with Web Learning
|
|
```bash
|
|
/project:infinite-web specs/d3_visualization_progressive.md d3_output 1
|
|
```
|
|
|
|
**What happens:**
|
|
1. Reads D3 spec
|
|
2. Performs initial web priming (3-5 foundational URLs)
|
|
3. Selects one URL from foundation category
|
|
4. Fetches URL, learns technique
|
|
5. Generates `d3_viz_1.html` applying learning
|
|
6. Documents web source in file footer
|
|
|
|
#### 2. Small Batch (5 Visualizations)
|
|
```bash
|
|
/project:infinite-web specs/d3_visualization_progressive.md d3_output 5
|
|
```
|
|
|
|
**What happens:**
|
|
1. Initial web priming phase
|
|
2. Launches 5 parallel agents simultaneously
|
|
3. Each agent gets different URL from foundation category
|
|
4. Each agent fetches, learns, generates
|
|
5. Produces 5 unique visualizations (d3_viz_1 through d3_viz_5)
|
|
6. Each demonstrates different technique from different web source
|
|
|
|
#### 3. Progressive Learning (20 Iterations)
|
|
```bash
|
|
/project:infinite-web specs/d3_visualization_progressive.md d3_output 20 specs/d3_url_strategy.json
|
|
```
|
|
|
|
**What happens:**
|
|
1. Initial web priming
|
|
2. Launches in batches of 5 agents
|
|
3. Iterations 1-5: Foundation URLs (basic concepts)
|
|
4. Iterations 6-12: Intermediate URLs (common patterns)
|
|
5. Iterations 13-20: Advanced URLs (complex techniques)
|
|
6. Each batch builds upon previous knowledge
|
|
7. Progressive sophistication across all 20 outputs
|
|
|
|
#### 4. Infinite Mode - Continuous Learning
|
|
```bash
|
|
/project:infinite-web specs/d3_visualization_progressive.md d3_infinite infinite specs/d3_url_strategy.json
|
|
```
|
|
|
|
**What happens:**
|
|
1. Initial web priming
|
|
2. Wave 1: 3-4 agents, foundation URLs
|
|
3. Wave 2: 3-4 agents, intermediate URLs
|
|
4. Wave 3: 3-4 agents, advanced URLs
|
|
5. Wave 4+: Expert URLs, then dynamic web search
|
|
6. Continues until context limits approached
|
|
7. Each wave more sophisticated than last
|
|
8. Graceful conclusion with summary
|
|
|
|
## How It Works
|
|
|
|
### Phase 0: Initial Web Priming
|
|
|
|
Before any generation, the system builds foundational knowledge:
|
|
|
|
```
|
|
1. Read specification file
|
|
2. Load URL strategy (if provided)
|
|
3. Fetch 3-5 priming URLs:
|
|
- Official documentation
|
|
- Tutorials
|
|
- Pattern libraries
|
|
- Best practices
|
|
4. Synthesize core concepts, patterns, principles
|
|
5. Build mental model for domain
|
|
```
|
|
|
|
### Phase 1-3: Analysis & Planning
|
|
|
|
```
|
|
1. Analyze specification with priming knowledge
|
|
2. Scan output directory for existing iterations
|
|
3. Track which URLs have been used (avoid duplicates)
|
|
4. Plan URL assignments for upcoming iterations
|
|
5. Match iteration numbers to URL difficulty levels
|
|
```
|
|
|
|
### Phase 4: Parallel Web-Enhanced Agents
|
|
|
|
Each sub-agent follows this process:
|
|
|
|
```
|
|
SUB-AGENT WORKFLOW:
|
|
├─ Receive assignment (iteration number + specific URL)
|
|
├─ WebFetch assigned URL
|
|
├─ Analyze content for techniques
|
|
├─ Extract 1-3 specific learnings
|
|
├─ Read specification
|
|
├─ Review existing iterations
|
|
├─ Generate output applying web learning
|
|
├─ Document web source and improvements
|
|
└─ Save file
|
|
```
|
|
|
|
**Key Features:**
|
|
- Each agent gets unique URL (no duplicates in batch)
|
|
- Parallel execution for speed
|
|
- Independent web research per agent
|
|
- All learnings documented in output
|
|
|
|
### Phase 5: Infinite Mode Waves
|
|
|
|
```
|
|
WAVE CYCLE:
|
|
while (context_available):
|
|
├─ Determine wave size (3-4 agents)
|
|
├─ Select URL category (foundation → expert)
|
|
├─ Assign unique URLs to each agent
|
|
├─ Launch parallel agents
|
|
├─ Monitor completion
|
|
├─ Update used URLs tracker
|
|
├─ Check context capacity
|
|
└─ Prepare next wave (if capacity available)
|
|
```
|
|
|
|
**Progressive Difficulty:**
|
|
- Early waves: Foundational concepts
|
|
- Mid waves: Common patterns and techniques
|
|
- Later waves: Advanced and expert topics
|
|
- Final waves: Dynamic web search for novel techniques
|
|
|
|
## URL Selection Strategies
|
|
|
|
### 1. Pre-defined URLs (Structured Learning)
|
|
|
|
**Advantages:**
|
|
- Curated, high-quality sources
|
|
- Progressive difficulty curve
|
|
- Predictable learning pathway
|
|
- Avoids low-quality content
|
|
|
|
**How it works:**
|
|
```json
|
|
// URL strategy file defines ranges
|
|
{
|
|
"foundation_urls": {
|
|
"iteration_range": [1, 5],
|
|
"urls": [
|
|
{"url": "...", "topic": "Basic Bar Charts"},
|
|
{"url": "...", "topic": "Scale Functions"},
|
|
...
|
|
]
|
|
}
|
|
}
|
|
```
|
|
|
|
Agent for iteration 3 gets URL from foundation_urls category.
|
|
|
|
### 2. Dynamic Web Search (Adaptive Discovery)
|
|
|
|
**Advantages:**
|
|
- Finds latest resources
|
|
- Adapts to specific needs
|
|
- Never runs out of sources
|
|
- Discovers novel techniques
|
|
|
|
**How it works:**
|
|
```
|
|
Agent determines knowledge gap
|
|
→ Generates targeted search query
|
|
→ Uses WebSearch tool
|
|
→ Selects best result
|
|
→ Fetches and learns from it
|
|
```
|
|
|
|
**Example query generation:**
|
|
```
|
|
"D3.js force simulation collision detection site:observablehq.com"
|
|
"D3 responsive design best practices 2024"
|
|
"Advanced D3 network visualization techniques"
|
|
```
|
|
|
|
### 3. Hybrid Mode (Recommended)
|
|
|
|
**Best of both worlds:**
|
|
- Use pre-defined URLs for core curriculum
|
|
- Fall back to dynamic search when URLs exhausted
|
|
- Dynamic search for novel explorations
|
|
- Structured foundation + adaptive innovation
|
|
|
|
## Creating Your Own Web-Enhanced Loop
|
|
|
|
### Step 1: Define Your Domain
|
|
|
|
Create a specification file for your domain:
|
|
|
|
**Template:** `specs/your_domain_progressive.md`
|
|
|
|
Key sections:
|
|
- Core challenge and goals
|
|
- Output format requirements
|
|
- Progressive enhancement dimensions
|
|
- Web research integration strategy
|
|
- Quality standards
|
|
- Iteration evolution pattern
|
|
|
|
### Step 2: Curate URL Strategy (Optional but Recommended)
|
|
|
|
Create URL strategy JSON: `specs/your_domain_url_strategy.json`
|
|
|
|
**Organize URLs by:**
|
|
- Priming (foundational knowledge)
|
|
- Foundation (basic techniques)
|
|
- Intermediate (common patterns)
|
|
- Advanced (complex implementations)
|
|
- Expert (cutting-edge techniques)
|
|
|
|
**Include:**
|
|
- Web search templates for dynamic discovery
|
|
- Topic tags for each URL
|
|
- Learning objectives
|
|
|
|
### Step 3: Run Your Loop
|
|
|
|
```bash
|
|
/project:infinite-web specs/your_domain_progressive.md output_dir <count> specs/your_domain_url_strategy.json
|
|
```
|
|
|
|
## Example Domains Beyond D3
|
|
|
|
### 1. React Component Library
|
|
```bash
|
|
# Spec: Progressive React component patterns
|
|
# URLs: React docs, popular libraries, best practices
|
|
# Output: Increasingly sophisticated React components
|
|
/project:infinite-web specs/react_components_progressive.md react_output 20
|
|
```
|
|
|
|
### 2. CSS Animation Techniques
|
|
```bash
|
|
# Spec: CSS animation mastery
|
|
# URLs: CSS-Tricks, MDN, CodePen examples
|
|
# Output: Progressive animation techniques
|
|
/project:infinite-web specs/css_animations_progressive.md css_output infinite
|
|
```
|
|
|
|
### 3. Python Data Science
|
|
```bash
|
|
# Spec: Data analysis and visualization
|
|
# URLs: Pandas docs, Matplotlib gallery, Seaborn tutorials
|
|
# Output: Jupyter notebooks with progressive techniques
|
|
/project:infinite-web specs/python_datascience_progressive.md notebooks 15
|
|
```
|
|
|
|
### 4. Machine Learning Models
|
|
```bash
|
|
# Spec: ML model implementations
|
|
# URLs: TensorFlow tutorials, papers, Kaggle kernels
|
|
# Output: Progressive ML implementations
|
|
/project:infinite-web specs/ml_models_progressive.md models 10
|
|
```
|
|
|
|
## Quality Assurance
|
|
|
|
### What Makes a Good Iteration?
|
|
|
|
**Web Integration:**
|
|
- ✅ Specific URL documented in output
|
|
- ✅ Technique from URL clearly identified
|
|
- ✅ Learning demonstrably applied (not just mentioned)
|
|
- ✅ Improvement over previous iteration explained
|
|
|
|
**Technical Quality:**
|
|
- ✅ Follows specification format exactly
|
|
- ✅ Code is clean, commented, functional
|
|
- ✅ Demonstrates mastery of learned technique
|
|
- ✅ Builds upon previous iterations appropriately
|
|
|
|
**Uniqueness:**
|
|
- ✅ No duplicate URLs across iterations
|
|
- ✅ Novel implementation (not copy-paste from web)
|
|
- ✅ Genuine advancement over previous work
|
|
- ✅ Creative synthesis of multiple learnings
|
|
|
|
### Common Pitfalls to Avoid
|
|
|
|
**Superficial Web Integration:**
|
|
- ❌ URL cited but technique not applied
|
|
- ❌ Copying example code without understanding
|
|
- ❌ Mentioning learning without demonstrating it
|
|
|
|
**Poor Progression:**
|
|
- ❌ Iterations don't build on each other
|
|
- ❌ No measurable improvement
|
|
- ❌ Duplicate techniques from different sources
|
|
|
|
**URL Management:**
|
|
- ❌ Reusing same URL in multiple iterations
|
|
- ❌ Low-quality or irrelevant sources
|
|
- ❌ Ignoring URL strategy difficulty curve
|
|
|
|
## Advanced Techniques
|
|
|
|
### Custom Web Search Strategies
|
|
|
|
Define search templates in your URL strategy:
|
|
|
|
```json
|
|
{
|
|
"web_search_templates": [
|
|
"{domain} {technique} tutorial site:observablehq.com",
|
|
"{domain} {feature} best practices 2024",
|
|
"advanced {domain} {area} technique",
|
|
"{domain} performance optimization {context}"
|
|
]
|
|
}
|
|
```
|
|
|
|
Agents can fill templates dynamically based on needs.
|
|
|
|
### Multi-Domain Knowledge Fusion
|
|
|
|
Combine multiple URL strategies:
|
|
|
|
```bash
|
|
# Fuse D3 + Design + Accessibility knowledge
|
|
/project:infinite-web specs/accessible_dataviz.md output 20 specs/multi_domain_urls.json
|
|
```
|
|
|
|
### Iteration Forking
|
|
|
|
Generate multiple variants per iteration:
|
|
|
|
```bash
|
|
# Each iteration explores 3 different techniques from same URL
|
|
/project:infinite-web specs/exploration_spec.md variants 30
|
|
```
|
|
|
|
### Knowledge Checkpointing
|
|
|
|
Summarize learnings periodically:
|
|
|
|
```
|
|
Iterations 1-10 → Generate summary.md with accumulated knowledge
|
|
Iterations 11-20 → Build upon summary + new research
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### WebFetch Failures
|
|
|
|
**Problem:** URL fetch fails
|
|
|
|
**Solutions:**
|
|
1. System falls back to WebSearch for topic
|
|
2. Tries alternative URL from same category
|
|
3. Logs failure and continues with fallback
|
|
|
|
### URL Exhaustion
|
|
|
|
**Problem:** All pre-defined URLs used
|
|
|
|
**Solutions:**
|
|
1. Automatically switches to dynamic WebSearch mode
|
|
2. Generates targeted queries based on:
|
|
- Current sophistication level
|
|
- Knowledge gaps
|
|
- Novel areas to explore
|
|
|
|
### Context Limits in Infinite Mode
|
|
|
|
**Problem:** Approaching context window limits
|
|
|
|
**Solutions:**
|
|
1. System monitors context usage continuously
|
|
2. Completes current wave before limits
|
|
3. Generates summary of progress
|
|
4. Graceful conclusion with statistics
|
|
|
|
### Poor Web Learning Integration
|
|
|
|
**Problem:** Agents cite but don't apply learnings
|
|
|
|
**Solutions:**
|
|
1. Strengthen integration directive in prompts
|
|
2. Review output for genuine technique application
|
|
2. Require code examples demonstrating learning
|
|
4. Validate improvement over previous iteration
|
|
|
|
## Best Practices
|
|
|
|
### 1. Start Small, Scale Up
|
|
```bash
|
|
# Test with 1 iteration first
|
|
/project:infinite-web specs/your_spec.md test 1
|
|
|
|
# Then try batch
|
|
/project:infinite-web specs/your_spec.md test 5
|
|
|
|
# Finally go infinite
|
|
/project:infinite-web specs/your_spec.md production infinite
|
|
```
|
|
|
|
### 2. Curate Quality URLs
|
|
- Choose authoritative sources
|
|
- Verify URLs are accessible
|
|
- Organize by genuine difficulty progression
|
|
- Include diverse perspectives
|
|
|
|
### 3. Write Clear Specifications
|
|
- Explicit output format requirements
|
|
- Clear quality standards
|
|
- Specific integration directives
|
|
- Progressive enhancement dimensions
|
|
|
|
### 4. Monitor First Wave
|
|
- Check first 3-5 outputs carefully
|
|
- Verify web learning application
|
|
- Adjust spec if needed
|
|
- Refine URL strategy based on results
|
|
|
|
### 5. Document Learnings
|
|
- Each iteration should document:
|
|
- Web source URL
|
|
- Specific technique learned
|
|
- How it was applied
|
|
- Improvement over previous iteration
|
|
|
|
## System Architecture Summary
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ /project:infinite-web Command │
|
|
│ ├─ Phase 0: Initial Web Priming (3-5 foundational URLs)│
|
|
│ ├─ Phase 1: Spec Analysis + Priming Knowledge │
|
|
│ ├─ Phase 2: Directory Recon + URL Tracking │
|
|
│ ├─ Phase 3: Web-Enhanced Iteration Strategy │
|
|
│ ├─ Phase 4: Parallel Agent Coordination │
|
|
│ │ ├─ Agent 1: URL A → Fetch → Learn → Generate │
|
|
│ │ ├─ Agent 2: URL B → Fetch → Learn → Generate │
|
|
│ │ └─ Agent N: URL N → Fetch → Learn → Generate │
|
|
│ └─ Phase 5: Infinite Mode Waves (optional) │
|
|
│ └─ Wave 1, 2, 3... until context limits │
|
|
└─────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Example Session
|
|
|
|
```bash
|
|
# Terminal session
|
|
$ claude
|
|
|
|
> /project:infinite-web specs/d3_visualization_progressive.md d3_viz 5 specs/d3_url_strategy.json
|
|
|
|
# System performs:
|
|
# 1. Reads D3 spec
|
|
# 2. Loads URL strategy
|
|
# 3. Fetches 3 priming URLs:
|
|
# - d3js.org/what-is-d3
|
|
# - observablehq.com/@d3/learn-d3
|
|
# - d3-graph-gallery.com
|
|
# 4. Synthesizes foundational knowledge
|
|
# 5. Launches 5 parallel agents:
|
|
# - Agent 1: d3js.org/d3-selection → learns selections → generates d3_viz_1.html
|
|
# - Agent 2: d3js.org/d3-scale → learns scales → generates d3_viz_2.html
|
|
# - Agent 3: observablehq.com/@d3/bar-chart → learns bar charts → d3_viz_3.html
|
|
# - Agent 4: observablehq.com/@d3/line-chart → learns line charts → d3_viz_4.html
|
|
# - Agent 5: observablehq.com/@d3/scatterplot → learns scatter → d3_viz_5.html
|
|
# 6. Each file documents its web source and learnings
|
|
# 7. Summary of all 5 iterations and techniques learned
|
|
|
|
✓ Generated 5 D3 visualizations with progressive web learning
|
|
✓ URLs used: 8 total (3 priming + 5 iteration)
|
|
✓ Techniques applied: selections, scales, bar charts, line charts, scatter plots
|
|
```
|
|
|
|
## Next Steps
|
|
|
|
1. **Try the D3 Example**: Run a small batch to see the system in action
|
|
2. **Create Your Domain**: Define spec and URL strategy for your use case
|
|
3. **Experiment with Modes**: Try single, batch, and infinite modes
|
|
4. **Refine Strategy**: Adjust URL difficulty progression based on results
|
|
5. **Scale Up**: Once validated, run larger batches or infinite mode
|
|
|
|
## Conclusion
|
|
|
|
The Web-Enhanced Infinite Agentic Loop transforms the original loop pattern into a **self-improving knowledge system**. Each iteration doesn't just generate output—it learns from the web, applies new techniques, and builds upon accumulated knowledge.
|
|
|
|
**Key Innovation:** Progressive web-based learning creates outputs that continuously improve in sophistication, quality, and technical excellence.
|
|
|
|
Start experimenting with D3 visualizations, then apply this pattern to any domain where progressive web-enhanced learning can create increasingly sophisticated outputs!
|