# Progressive SDG Network Visualization Specification

## Core Challenge
Create **progressively sophisticated SDG network visualizations** that integrate multiple open data APIs to display environmental, scientific, and sustainable development data as interactive force-directed graphs. Each iteration should discover new data sources and enhance visual/interactive capabilities.

## Output Requirements

**File Naming**: `sdg_viz_[iteration_number].html`

**Content Structure**: Self-contained HTML file with embedded D3.js visualization
```html
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>SDG Network Visualization [Iteration Number] - [Data Source Theme]</title>
    <style>
        /* Complete styling for force-directed graph */
        /* Node styling: colors, borders, sizes, labels */
        /* Edge styling: colors, widths, strengths, patterns */
        /* Interactive controls and UI elements */
        /* Responsive design for all screen sizes */
    </style>
</head>
<body>
    <div id="visualization-container">
        <!-- Main force-directed graph canvas -->
        <svg id="network-graph"></svg>

        <!-- Interactive controls -->
        <div id="controls">
            <!-- Filters, search, zoom controls -->
        </div>

        <!-- Information panel -->
        <div id="info-panel">
            <!-- Node/edge details, data source attribution -->
        </div>

        <!-- Legend -->
        <div id="legend">
            <!-- Color coding, node types, edge meanings -->
        </div>
    </div>

    <!-- D3.js library -->
    <script src="https://d3js.org/d3.v7.min.js"></script>

    <script>
        // API data fetching and integration
        // Force-directed graph implementation
        // Interactive behaviors and animations
        // Data transformation and network construction
    </script>

    <!-- Web Source Attribution -->
    <footer>
        <p><strong>Data Sources:</strong> [List of APIs used]</p>
        <p><strong>Web Learning Source:</strong> [URL researched for this iteration]</p>
        <p><strong>Techniques Applied:</strong> [Specific learnings from web source]</p>
    </footer>
</body>
</html>
```

## Progressive Enhancement Dimensions

### **Phase 1: Foundation (Iterations 1-5)**
**Focus**: Basic network visualization with single API integration

**Node Enhancements:**
- Size based on data properties (e.g., population, magnitude)
- Color coding by category/type (using color scales)
- Clear labels with smart positioning
- Hover tooltips with basic information

**Edge Enhancements:**
- Width based on relationship strength
- Color coding for relationship types
- Straight lines with arrows for directionality
- Dashed/solid patterns for different relationship classes

**Interactivity:**
- Drag nodes to reposition
- Zoom and pan controls
- Click to highlight connected nodes
- Basic filtering by node type

**Data Integration:**
- Single API source per iteration
- Simple JSON data transformation
- Basic node-edge relationship mapping
- Manual API endpoint definition

### **Phase 2: Intermediate (Iterations 6-12)**
**Focus**: Multi-source integration and enhanced interactivity

**Node Enhancements:**
- Dynamic sizing with min/max constraints
- Multi-property color encoding (fill + border)
- Icon/symbol overlays for node types
- Rich tooltips with formatted data tables
- Node clustering and grouping
- Custom shapes for different entity types

**Edge Enhancements:**
- Curved edges for better visibility
- Dynamic edge strength from data properties
- Animated flow along edges
- Edge bundling for cleaner layouts
- Multiple edge types with different styles

**Interactivity:**
- Search functionality to find nodes
- Multi-select for comparison
- Time-based filtering/animation
- Expandable/collapsible node groups
- Side panel with detailed statistics
- Export data/visualization options

**Data Integration:**
- Combine 2-3 API sources
- Automatic data merging by key fields
- API discovery via Swagger/OpenAPI specs
- Error handling for API failures
- Caching for performance

### **Phase 3: Advanced (Iterations 13-20)**
**Focus**: Sophisticated algorithms and real-time capabilities

**Node Enhancements:**
- Force-based clustering algorithms
- Community detection visualization
- Centrality measures (betweenness, closeness)
- Node importance ranking
- Multi-level node hierarchies
- Animated node state transitions

**Edge Enhancements:**
- Weighted force simulation
- Edge confidence/uncertainty visualization
- Temporal edge evolution
- Hierarchical edge bundling
- Edge crossing minimization

**Interactivity:**
- Real-time data updates
- Graph layout algorithm selection
- Advanced filtering with boolean logic
- Shortest path highlighting
- Neighborhood exploration
- Graph metric dashboards
- Collaborative filtering

**Data Integration:**
- 4+ API sources integrated
- Automated API endpoint discovery
- Data quality indicators
- Cross-API entity resolution
- Streaming data integration
- Webhook support for updates

### **Phase 4: Expert (Iterations 21+)**
**Focus**: Novel techniques and AI-enhanced features

**Node Enhancements:**
- ML-based node positioning
- Anomaly detection highlighting
- Predictive node attributes
- Semantic node embedding
- Dynamic node importance
- Multi-dimensional projection

**Edge Enhancements:**
- Probabilistic edge rendering
- Predicted relationship suggestions
- Edge importance scoring
- Dynamic edge weight learning
- Causal relationship inference

**Interactivity:**
- Natural language queries
- AI-powered graph insights
- Automatic pattern detection
- Interactive graph queries (Cypher-like)
- Graph comparison tools
- VR/AR integration capabilities

**Data Integration:**
- Federated query across APIs
- Automatic schema alignment
- Knowledge graph construction
- Semantic web integration (RDF, SPARQL)
- Real-time distributed data
- Blockchain data sources

## API Discovery Strategy

### **Target Data Domains**
Each iteration should discover and integrate APIs from these domains:

**Environmental Data:**
- Air quality (OpenAQ, EPA APIs)
- Water quality (USGS, EPA Water)
- Climate data (NOAA, NASA)
- Weather (OpenWeatherMap, Weather.gov)
- Ocean data (NOAA Ocean, Marine APIs)

**Scientific Data:**
- Biodiversity (GBIF, iNaturalist)
- Forest data (Global Forest Watch)
- Soil data (SoilGrids, ISRIC)
- Satellite imagery (NASA Earth, Sentinel)
- Seismic data (USGS Earthquake)

**Sustainable Development:**
- UN SDG API (existing baseline)
- World Bank Open Data
- OECD Statistics
- WHO Global Health
- FAO Agricultural Data

**Discovery Process:**
1. **Web Search for APIs**: Use WebSearch to find: `"[domain] API swagger site:github.com"`
2. **Swagger/OpenAPI Analysis**: Parse API specs to find endpoints
3. **Data Quality Check**: Verify API has relevant network-ready data
4. **Integration Planning**: Map API data to nodes and edges
5. **Documentation**: Record API details in visualization

### **API Selection Criteria**
- **Open Access**: Free or freemium tier available
- **REST/JSON**: Easy integration with JavaScript
- **Documentation**: Well-documented endpoints
- **Reliability**: Active maintenance and uptime
- **Data Richness**: Complex enough for network visualization
- **SDG Relevance**: Connects to sustainability goals

## Network Data Model

### **Node Schema**
```javascript
{
    id: "unique_identifier",
    name: "Display Name",
    type: "category", // e.g., "country", "indicator", "species"
    group: "group_name", // for color coding
    size: numeric_value, // for visual sizing
    properties: {
        // Domain-specific attributes
        value: number,
        unit: "string",
        timestamp: "ISO_date",
        source_api: "api_name",
        // ... additional properties
    },
    x: number, // force layout position
    y: number
}
```

### **Edge Schema**
```javascript
{
    source: "node_id",
    target: "node_id",
    type: "relationship_type",
    strength: numeric_value, // for force simulation
    weight: numeric_value, // for visual width
    properties: {
        correlation: number,
        confidence: number,
        source_api: "api_name",
        // ... additional properties
    }
}
```

## Web Learning Integration

### **Each Iteration Must:**
1. **Fetch Assigned URL**: Use WebFetch to retrieve documentation/tutorial
2. **Extract Techniques**: Identify 1-3 specific techniques from the source
3. **Apply Learning**: Implement the technique in the visualization
4. **Document Source**: Clearly attribute the web source and what was learned
5. **Demonstrate Improvement**: Show measurable enhancement from previous iteration

### **Web Source Categories**

**D3.js Techniques:**
- Observable HQ notebooks (d3.js examples)
- D3.js official documentation
- Mike Bostock's blocks
- D3 Graph Gallery
- Interactive graph tutorials

**Force-Directed Graphs:**
- Force simulation parameters
- Collision detection
- Link distance functions
- Clustering algorithms
- Layout optimization

**Data Integration:**
- REST API patterns
- CORS handling
- Async data loading
- Data transformation pipelines
- Error handling strategies

**Visual Design:**
- Color theory for networks
- Graph readability principles
- Interactive UX patterns
- Animation best practices
- Accessibility in visualizations

## Quality Standards

### **Functional Requirements**
- ✅ Fetches data from at least one open API
- ✅ Renders as force-directed graph with D3.js
- ✅ Nodes and edges have meaningful properties from data
- ✅ Interactive (hover, click, drag minimum)
- ✅ Responsive design works on desktop and mobile
- ✅ Clear data source attribution
- ✅ Error handling for API failures

### **Visual Excellence**
- ✅ Professional color scheme with good contrast
- ✅ Readable labels and legends
- ✅ Smooth animations (60fps)
- ✅ Clear visual hierarchy
- ✅ Consistent styling throughout
- ✅ Loading states for async operations

### **Technical Quality**
- ✅ Self-contained HTML file (or minimal external deps)
- ✅ Clean, commented JavaScript code
- ✅ Efficient force simulation (no lag)
- ✅ Proper error handling and fallbacks
- ✅ Browser compatibility (modern browsers)
- ✅ No console errors

### **Progressive Learning**
- ✅ Demonstrates new technique from web source
- ✅ Builds upon previous iterations where appropriate
- ✅ Documents what was learned and how applied
- ✅ Shows measurable improvement
- ✅ Explores new API sources

## Iteration Evolution Strategy

### **Starting Point (Iteration 1)**
- Use UN SDG API (baseline from existing project)
- Basic force-directed graph with D3.js v7
- Simple node/edge styling
- Basic hover tooltips
- Learn from: D3.js force simulation documentation

### **Early Iterations (2-5)**
- Add one new API per iteration
- Focus on core graph enhancements
- Build interactive controls
- Learn: Color scales, tooltips, filtering

### **Middle Iterations (6-12)**
- Combine multiple APIs
- Advanced interactivity
- Custom layouts and algorithms
- Learn: Graph algorithms, data merging, UX patterns

### **Advanced Iterations (13-20)**
- Real-time data integration
- Complex algorithms (community detection, centrality)
- Rich dashboards
- Learn: Advanced D3, graph theory, streaming data

### **Expert Iterations (21+)**
- AI/ML integration
- Novel visualization techniques
- Cross-platform capabilities
- Learn: Cutting-edge graph visualization research

## Success Metrics

Each iteration should improve on at least 3 of these metrics:

**Data Richness:**
- Number of API sources integrated
- Total nodes in network
- Total edges in network
- Data update frequency

**Visual Quality:**
- Color coding sophistication
- Label readability score
- Animation smoothness
- Layout clarity

**Interactivity:**
- Number of interactive features
- Response time to user actions
- Filter/search capabilities
- Information density

**Technical Innovation:**
- New techniques applied
- Performance improvements
- Code quality metrics
- Browser compatibility

## Ultra-Thinking Directive

Before each iteration, deeply consider:

**API Discovery:**
- What domain provides the most value for SDG insights?
- Which APIs have network-ready data structures?
- How can multiple APIs be meaningfully combined?
- What data relationships would be most revealing?

**Visualization Strategy:**
- What graph layout best reveals the data patterns?
- How can node/edge properties enhance understanding?
- What interactivity would provide the most insight?
- How does this build on previous iterations?

**Web Learning Application:**
- What specific technique from the web source applies here?
- How can this technique improve the visualization?
- What adaptations are needed for our use case?
- How does this advance the state of the art?

**User Value:**
- What insights does this visualization reveal?
- How intuitive is the interaction model?
- What questions can users answer with this tool?
- How does this serve SDG analysis and understanding?

**Generate visualizations that:**
- **Discover New Data**: Find and integrate novel API sources
- **Apply Web Knowledge**: Demonstrate clear learning from web research
- **Enhance Progressively**: Each iteration measurably better than the last
- **Serve SDGs**: Provide genuine value for sustainable development insights
- **Inspire Exploration**: Create compelling, informative, interactive experiences
