infinite-agents-public/SDG_NETWORK_GUIDE.md

SDG Network Visualization - Web-Enhanced Infinite Loop Guide

Overview

This guide explains how to use the web-enhanced infinite agentic loop to generate progressively sophisticated SDG (Sustainable Development Goals) network visualizations that integrate multiple open data APIs.

What Gets Generated

Each iteration produces a self-contained HTML file with:

• Force-directed network graph using D3.js
• Real data from open APIs (environmental, scientific, SDG-related)
• Progressive enhancements in visualization quality and interactivity
• Web-sourced techniques applied and documented
• API discovery through automated web searches

Current Baseline

The existing SDG network visualization (ai_docs/sdg-network-modelling/) provides:

• Basic D3.js v3 force-directed graph
• UN SDG API integration
• Simple node/edge rendering
• Basic hover tooltips
• Geographic and SDG goal relationships

Progressive Enhancement Strategy

Phase 1: Foundation (Iterations 1-5)

Visual Enhancements:

• Node size based on data magnitude
• Color coding by categories (using D3 color scales)
• Smart label positioning
• Rich hover tooltips with formatted data
• Edge width based on relationship strength
• Directional arrows and edge patterns

Interactivity:

• Drag nodes to reposition
• Zoom and pan controls
• Click to highlight connected nodes
• Basic filtering by node type
• Legend for color coding

Data:

• Single API per iteration
• Examples: UN SDG API, World Bank, OpenAQ, USGS Water

Phase 2: Intermediate (Iterations 6-12)

Visual Enhancements:

• Multi-property color encoding (fill color + border color)
• Icon overlays on nodes
• Curved edges for better visibility
• Node clustering by community
• Edge bundling for cleaner layouts
• Custom node shapes for different entity types

Interactivity:

• Search functionality to find nodes
• Multi-select for comparison
• Time-based animation
• Expandable/collapsible groups
• Side panel with statistics
• Export visualization and data

Data:

• Combine 2-3 API sources
• Automatic data merging
• Examples: NASA + NOAA + EPA, WHO + World Bank + UN

Phase 3: Advanced (Iterations 13-20)

Visual Enhancements:

• Force-based clustering algorithms
• Community detection visualization
• Centrality measures displayed
• Node importance ranking
• Hierarchical edge bundling
• Animated state transitions

Interactivity:

• Real-time data updates
• Graph layout algorithm switching
• Advanced filtering (boolean logic)
• Shortest path finding
• Neighborhood exploration
• Graph metric dashboards

Data:

• 4+ API sources integrated
• Automated API discovery via Swagger search
• Cross-API entity resolution
• Streaming data integration

Phase 4: Expert (Iterations 21+)

Visual Enhancements:

• ML-based optimal node positioning
• Anomaly detection highlighting
• Predictive attributes visualization
• 3D network projections
• VR/AR capabilities

Interactivity:

• Natural language graph queries
• AI-powered insights
• Automatic pattern detection
• Graph comparison tools
• Collaborative features

Data:

• Federated queries across APIs
• Knowledge graph construction
• Semantic web integration
• Blockchain data sources

API Discovery Process

Each iteration automatically discovers new data sources:

Target Domains:

1. Environmental: Air quality, water quality, climate
2. Scientific: Biodiversity, satellites, research datasets
3. Atmospheric: Weather, air monitoring
4. Ecological: Forests, species, conservation
5. Ocean: Marine life, coastal data
6. Geophysical: Earthquakes, volcanoes
7. Health: Disease, mortality, healthcare
8. SDG: Development indicators

Discovery Method:

1. Web search: "[domain] API swagger site:github.com"
2. Parse Swagger/OpenAPI specifications
3. Evaluate API quality and relevance
4. Map data to network structure (nodes/edges)
5. Integrate and visualize

API Examples:

• UN SDG API: https://unstats.un.org/SDGAPI/v1/sdg/
• OpenAQ: Air quality data worldwide
• USGS Water Services: Water quality and flow
• Global Forest Watch: Deforestation and forest data
• NASA Earthdata: Satellite and earth science
• NOAA: Weather, ocean, climate data
• GBIF: Biodiversity occurrence data
• World Bank: Development indicators

Web Learning Integration

Each iteration learns from curated web resources:

Foundation Topics:

• D3.js fundamentals and data binding
• Basic force-directed graphs
• Color scales and node styling
• Interactive tooltips and zoom
• Canvas rendering for performance

Intermediate Topics:

• Force-based clustering
• Custom force implementations
• Advanced color schemes
• Animated transitions
• Modular component architecture

Advanced Topics:

• Edge bundling algorithms
• 3D projections and rotations
• Adaptive rendering for large graphs
• Real-time streaming data
• Multi-view interfaces

Expert Topics:

• Graph theory algorithms (centrality, shortest path)
• Community detection
• Semantic zoom
• Export capabilities
• ML-enhanced layouts

Usage Examples

Single Iteration with Default Settings:

claude
> /project:infinite-web specs/sdg_network_progressive.md sdg_viz 1

This generates sdg_viz/sdg_viz_1.html with:

• UN SDG API baseline
• Basic force graph from D3.js documentation
• Foundation-level enhancements

Small Batch (5 iterations):

> /project:infinite-web specs/sdg_network_progressive.md sdg_viz 5

Generates 5 files with:

• Different API sources (UN SDG, World Bank, OpenAQ, etc.)
• Progressive techniques from foundation URLs
• Each builds upon insights from previous iterations

Medium Batch with Custom URL Strategy:

> /project:infinite-web specs/sdg_network_progressive.md sdg_viz 12 specs/sdg_network_url_strategy.json

Generates 12 iterations:

• Iterations 1-5: Foundation techniques
• Iterations 6-12: Intermediate techniques
• Multiple APIs combined
• Advanced interactivity

Infinite Mode (Continuous until context limit):

> /project:infinite-web specs/sdg_network_progressive.md sdg_viz infinite specs/sdg_network_url_strategy.json

Runs continuously with:

• Progressive difficulty waves
• Automatic API discovery via web search
• Increasingly sophisticated visualizations
• Stops gracefully at context limit

Expected Output Structure

Each sdg_viz_[N].html file contains:

<!DOCTYPE html>
<html>
<head>
    <title>SDG Network Visualization [N] - [Data Theme]</title>
    <style>/* Complete styling */</style>
</head>
<body>
    <div id="visualization-container">
        <svg id="network-graph"></svg>
        <div id="controls">/* Interactive controls */</div>
        <div id="info-panel">/* Data details */</div>
        <div id="legend">/* Visual legend */</div>
    </div>

    <script src="https://d3js.org/d3.v7.min.js"></script>
    <script>
        // Fetch data from API(s)
        // Build network graph
        // Render force-directed visualization
        // Add interactivity
    </script>

    <footer>
        <p><strong>Data Sources:</strong> [APIs used]</p>
        <p><strong>Web Learning:</strong> [URL + techniques]</p>
    </footer>
</body>
</html>

Key Improvements Over Baseline

Data Integration:

• ❌ Old: Single hardcoded API (UN SDG only)
• ✅ New: Automatic API discovery, multiple sources combined

Visualization:

• ❌ Old: Basic circles and lines, single color scheme
• ✅ New: Rich node/edge properties, dynamic sizing, multi-attribute encoding

Interactivity:

• ❌ Old: Simple hover tooltip
• ✅ New: Search, filter, zoom, click actions, side panels, exports

Learning:

• ❌ Old: Static implementation
• ✅ New: Each iteration learns from web and improves

Technology:

• ❌ Old: D3.js v3 (outdated)
• ✅ New: D3.js v7 (modern), latest techniques

Quality Metrics

Each iteration is evaluated on:

Data Richness:

• Number of API sources (1 → 5+)
• Total nodes/edges
• Data update frequency

Visual Quality:

• Color coding sophistication
• Label readability
• Animation smoothness (60fps)
• Layout clarity

Interactivity:

• Number of features (hover → search → filter → AI)
• Response time
• Information density

Technical Innovation:

• New techniques applied
• Performance improvements
• Code quality

Advanced Use Cases

Compare Algorithms:

Generate iterations with different force algorithms:

# Each iteration learns a different layout algorithm
> /project:infinite-web specs/sdg_network_progressive.md algorithm_comparison 8

Domain-Specific Networks:

Focus on specific data domains by modifying the spec:

# Ocean/marine data network
> /project:infinite-web specs/ocean_network_progressive.md ocean_viz 10

# Climate change network
> /project:infinite-web specs/climate_network_progressive.md climate_viz 15

Real-Time Dashboards:

Advanced iterations include streaming data:

# Generate real-time monitoring dashboards
> /project:infinite-web specs/sdg_network_progressive.md realtime_sdg 20

Troubleshooting

API Access Issues:

• Some APIs require keys → iterations handle gracefully with fallbacks
• CORS errors → iterations use proxy or alternatives
• Rate limits → iterations implement caching

Performance:

• Large graphs → iterations use Canvas or WebGL
• Slow rendering → iterations implement LOD and culling
• Memory issues → iterations use virtualization

Browser Compatibility:

• All iterations target modern browsers (Chrome, Firefox, Safari, Edge)
• Fallbacks for older browsers where reasonable
• Progressive enhancement ensures basic functionality

Next Steps

1. Run your first iteration:

   claude
   > /project:infinite-web specs/sdg_network_progressive.md sdg_viz 1
   

2. Review the output:

   • Open sdg_viz/sdg_viz_1.html in browser
   • Check data sources and web learning attribution
   • Test interactivity

3. Generate a batch:

   > /project:infinite-web specs/sdg_network_progressive.md sdg_viz 5
   

4. Compare iterations:

   • See progressive improvements
   • Note new techniques applied
   • Observe API diversity

5. Go infinite:

   > /project:infinite-web specs/sdg_network_progressive.md sdg_viz infinite
   

Resources

Specifications:

• specs/sdg_network_progressive.md - Main visualization spec
• specs/sdg_network_url_strategy.json - Progressive learning URLs

Commands:

• .claude/commands/infinite-web.md - Web-enhanced loop implementation

Baseline Reference:

• ai_docs/sdg-network-modelling/ - Original SDG network project

Generated Output:

• sdg_viz/ - Output directory for iterations (create via command)

Contributing

To extend this system:

1. Add new API sources to sdg_network_url_strategy.json search templates
2. Add learning resources as new URLs in the strategy file
3. Create domain-specific specs by copying and modifying the base spec
4. Share successful iterations as examples

Credits

• Original SDG Network: First Green Bank Network visualization
• D3.js: Michael Bostock and contributors
• Observable: Interactive notebook platform
• Open APIs: UN, World Bank, NOAA, USGS, NASA, and many others
• Claude Code: Infinite agentic loop implementation