447 lines
13 KiB
Markdown
447 lines
13 KiB
Markdown
# Progressive SDG Network Visualization Specification
|
|
|
|
## Core Challenge
|
|
Create **progressively sophisticated SDG network visualizations** that integrate multiple open data APIs to display environmental, scientific, and sustainable development data as interactive force-directed graphs. Each iteration should discover new data sources and enhance visual/interactive capabilities.
|
|
|
|
## Output Requirements
|
|
|
|
**File Naming**: `sdg_viz_[iteration_number].html`
|
|
|
|
**Content Structure**: Self-contained HTML file with embedded D3.js visualization
|
|
```html
|
|
<!DOCTYPE html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="UTF-8">
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
<title>SDG Network Visualization [Iteration Number] - [Data Source Theme]</title>
|
|
<style>
|
|
/* Complete styling for force-directed graph */
|
|
/* Node styling: colors, borders, sizes, labels */
|
|
/* Edge styling: colors, widths, strengths, patterns */
|
|
/* Interactive controls and UI elements */
|
|
/* Responsive design for all screen sizes */
|
|
</style>
|
|
</head>
|
|
<body>
|
|
<div id="visualization-container">
|
|
<!-- Main force-directed graph canvas -->
|
|
<svg id="network-graph"></svg>
|
|
|
|
<!-- Interactive controls -->
|
|
<div id="controls">
|
|
<!-- Filters, search, zoom controls -->
|
|
</div>
|
|
|
|
<!-- Information panel -->
|
|
<div id="info-panel">
|
|
<!-- Node/edge details, data source attribution -->
|
|
</div>
|
|
|
|
<!-- Legend -->
|
|
<div id="legend">
|
|
<!-- Color coding, node types, edge meanings -->
|
|
</div>
|
|
</div>
|
|
|
|
<!-- D3.js library -->
|
|
<script src="https://d3js.org/d3.v7.min.js"></script>
|
|
|
|
<script>
|
|
// API data fetching and integration
|
|
// Force-directed graph implementation
|
|
// Interactive behaviors and animations
|
|
// Data transformation and network construction
|
|
</script>
|
|
|
|
<!-- Web Source Attribution -->
|
|
<footer>
|
|
<p><strong>Data Sources:</strong> [List of APIs used]</p>
|
|
<p><strong>Web Learning Source:</strong> [URL researched for this iteration]</p>
|
|
<p><strong>Techniques Applied:</strong> [Specific learnings from web source]</p>
|
|
</footer>
|
|
</body>
|
|
</html>
|
|
```
|
|
|
|
## Progressive Enhancement Dimensions
|
|
|
|
### **Phase 1: Foundation (Iterations 1-5)**
|
|
**Focus**: Basic network visualization with single API integration
|
|
|
|
**Node Enhancements:**
|
|
- Size based on data properties (e.g., population, magnitude)
|
|
- Color coding by category/type (using color scales)
|
|
- Clear labels with smart positioning
|
|
- Hover tooltips with basic information
|
|
|
|
**Edge Enhancements:**
|
|
- Width based on relationship strength
|
|
- Color coding for relationship types
|
|
- Straight lines with arrows for directionality
|
|
- Dashed/solid patterns for different relationship classes
|
|
|
|
**Interactivity:**
|
|
- Drag nodes to reposition
|
|
- Zoom and pan controls
|
|
- Click to highlight connected nodes
|
|
- Basic filtering by node type
|
|
|
|
**Data Integration:**
|
|
- Single API source per iteration
|
|
- Simple JSON data transformation
|
|
- Basic node-edge relationship mapping
|
|
- Manual API endpoint definition
|
|
|
|
### **Phase 2: Intermediate (Iterations 6-12)**
|
|
**Focus**: Multi-source integration and enhanced interactivity
|
|
|
|
**Node Enhancements:**
|
|
- Dynamic sizing with min/max constraints
|
|
- Multi-property color encoding (fill + border)
|
|
- Icon/symbol overlays for node types
|
|
- Rich tooltips with formatted data tables
|
|
- Node clustering and grouping
|
|
- Custom shapes for different entity types
|
|
|
|
**Edge Enhancements:**
|
|
- Curved edges for better visibility
|
|
- Dynamic edge strength from data properties
|
|
- Animated flow along edges
|
|
- Edge bundling for cleaner layouts
|
|
- Multiple edge types with different styles
|
|
|
|
**Interactivity:**
|
|
- Search functionality to find nodes
|
|
- Multi-select for comparison
|
|
- Time-based filtering/animation
|
|
- Expandable/collapsible node groups
|
|
- Side panel with detailed statistics
|
|
- Export data/visualization options
|
|
|
|
**Data Integration:**
|
|
- Combine 2-3 API sources
|
|
- Automatic data merging by key fields
|
|
- API discovery via Swagger/OpenAPI specs
|
|
- Error handling for API failures
|
|
- Caching for performance
|
|
|
|
### **Phase 3: Advanced (Iterations 13-20)**
|
|
**Focus**: Sophisticated algorithms and real-time capabilities
|
|
|
|
**Node Enhancements:**
|
|
- Force-based clustering algorithms
|
|
- Community detection visualization
|
|
- Centrality measures (betweenness, closeness)
|
|
- Node importance ranking
|
|
- Multi-level node hierarchies
|
|
- Animated node state transitions
|
|
|
|
**Edge Enhancements:**
|
|
- Weighted force simulation
|
|
- Edge confidence/uncertainty visualization
|
|
- Temporal edge evolution
|
|
- Hierarchical edge bundling
|
|
- Edge crossing minimization
|
|
|
|
**Interactivity:**
|
|
- Real-time data updates
|
|
- Graph layout algorithm selection
|
|
- Advanced filtering with boolean logic
|
|
- Shortest path highlighting
|
|
- Neighborhood exploration
|
|
- Graph metric dashboards
|
|
- Collaborative filtering
|
|
|
|
**Data Integration:**
|
|
- 4+ API sources integrated
|
|
- Automated API endpoint discovery
|
|
- Data quality indicators
|
|
- Cross-API entity resolution
|
|
- Streaming data integration
|
|
- Webhook support for updates
|
|
|
|
### **Phase 4: Expert (Iterations 21+)**
|
|
**Focus**: Novel techniques and AI-enhanced features
|
|
|
|
**Node Enhancements:**
|
|
- ML-based node positioning
|
|
- Anomaly detection highlighting
|
|
- Predictive node attributes
|
|
- Semantic node embedding
|
|
- Dynamic node importance
|
|
- Multi-dimensional projection
|
|
|
|
**Edge Enhancements:**
|
|
- Probabilistic edge rendering
|
|
- Predicted relationship suggestions
|
|
- Edge importance scoring
|
|
- Dynamic edge weight learning
|
|
- Causal relationship inference
|
|
|
|
**Interactivity:**
|
|
- Natural language queries
|
|
- AI-powered graph insights
|
|
- Automatic pattern detection
|
|
- Interactive graph queries (Cypher-like)
|
|
- Graph comparison tools
|
|
- VR/AR integration capabilities
|
|
|
|
**Data Integration:**
|
|
- Federated query across APIs
|
|
- Automatic schema alignment
|
|
- Knowledge graph construction
|
|
- Semantic web integration (RDF, SPARQL)
|
|
- Real-time distributed data
|
|
- Blockchain data sources
|
|
|
|
## API Discovery Strategy
|
|
|
|
### **Target Data Domains**
|
|
Each iteration should discover and integrate APIs from these domains:
|
|
|
|
**Environmental Data:**
|
|
- Air quality (OpenAQ, EPA APIs)
|
|
- Water quality (USGS, EPA Water)
|
|
- Climate data (NOAA, NASA)
|
|
- Weather (OpenWeatherMap, Weather.gov)
|
|
- Ocean data (NOAA Ocean, Marine APIs)
|
|
|
|
**Scientific Data:**
|
|
- Biodiversity (GBIF, iNaturalist)
|
|
- Forest data (Global Forest Watch)
|
|
- Soil data (SoilGrids, ISRIC)
|
|
- Satellite imagery (NASA Earth, Sentinel)
|
|
- Seismic data (USGS Earthquake)
|
|
|
|
**Sustainable Development:**
|
|
- UN SDG API (existing baseline)
|
|
- World Bank Open Data
|
|
- OECD Statistics
|
|
- WHO Global Health
|
|
- FAO Agricultural Data
|
|
|
|
**Discovery Process:**
|
|
1. **Web Search for APIs**: Use WebSearch to find: `"[domain] API swagger site:github.com"`
|
|
2. **Swagger/OpenAPI Analysis**: Parse API specs to find endpoints
|
|
3. **Data Quality Check**: Verify API has relevant network-ready data
|
|
4. **Integration Planning**: Map API data to nodes and edges
|
|
5. **Documentation**: Record API details in visualization
|
|
|
|
### **API Selection Criteria**
|
|
- **Open Access**: Free or freemium tier available
|
|
- **REST/JSON**: Easy integration with JavaScript
|
|
- **Documentation**: Well-documented endpoints
|
|
- **Reliability**: Active maintenance and uptime
|
|
- **Data Richness**: Complex enough for network visualization
|
|
- **SDG Relevance**: Connects to sustainability goals
|
|
|
|
## Network Data Model
|
|
|
|
### **Node Schema**
|
|
```javascript
|
|
{
|
|
id: "unique_identifier",
|
|
name: "Display Name",
|
|
type: "category", // e.g., "country", "indicator", "species"
|
|
group: "group_name", // for color coding
|
|
size: numeric_value, // for visual sizing
|
|
properties: {
|
|
// Domain-specific attributes
|
|
value: number,
|
|
unit: "string",
|
|
timestamp: "ISO_date",
|
|
source_api: "api_name",
|
|
// ... additional properties
|
|
},
|
|
x: number, // force layout position
|
|
y: number
|
|
}
|
|
```
|
|
|
|
### **Edge Schema**
|
|
```javascript
|
|
{
|
|
source: "node_id",
|
|
target: "node_id",
|
|
type: "relationship_type",
|
|
strength: numeric_value, // for force simulation
|
|
weight: numeric_value, // for visual width
|
|
properties: {
|
|
correlation: number,
|
|
confidence: number,
|
|
source_api: "api_name",
|
|
// ... additional properties
|
|
}
|
|
}
|
|
```
|
|
|
|
## Web Learning Integration
|
|
|
|
### **Each Iteration Must:**
|
|
1. **Fetch Assigned URL**: Use WebFetch to retrieve documentation/tutorial
|
|
2. **Extract Techniques**: Identify 1-3 specific techniques from the source
|
|
3. **Apply Learning**: Implement the technique in the visualization
|
|
4. **Document Source**: Clearly attribute the web source and what was learned
|
|
5. **Demonstrate Improvement**: Show measurable enhancement from previous iteration
|
|
|
|
### **Web Source Categories**
|
|
|
|
**D3.js Techniques:**
|
|
- Observable HQ notebooks (d3.js examples)
|
|
- D3.js official documentation
|
|
- Mike Bostock's blocks
|
|
- D3 Graph Gallery
|
|
- Interactive graph tutorials
|
|
|
|
**Force-Directed Graphs:**
|
|
- Force simulation parameters
|
|
- Collision detection
|
|
- Link distance functions
|
|
- Clustering algorithms
|
|
- Layout optimization
|
|
|
|
**Data Integration:**
|
|
- REST API patterns
|
|
- CORS handling
|
|
- Async data loading
|
|
- Data transformation pipelines
|
|
- Error handling strategies
|
|
|
|
**Visual Design:**
|
|
- Color theory for networks
|
|
- Graph readability principles
|
|
- Interactive UX patterns
|
|
- Animation best practices
|
|
- Accessibility in visualizations
|
|
|
|
## Quality Standards
|
|
|
|
### **Functional Requirements**
|
|
- ✅ Fetches data from at least one open API
|
|
- ✅ Renders as force-directed graph with D3.js
|
|
- ✅ Nodes and edges have meaningful properties from data
|
|
- ✅ Interactive (hover, click, drag minimum)
|
|
- ✅ Responsive design works on desktop and mobile
|
|
- ✅ Clear data source attribution
|
|
- ✅ Error handling for API failures
|
|
|
|
### **Visual Excellence**
|
|
- ✅ Professional color scheme with good contrast
|
|
- ✅ Readable labels and legends
|
|
- ✅ Smooth animations (60fps)
|
|
- ✅ Clear visual hierarchy
|
|
- ✅ Consistent styling throughout
|
|
- ✅ Loading states for async operations
|
|
|
|
### **Technical Quality**
|
|
- ✅ Self-contained HTML file (or minimal external deps)
|
|
- ✅ Clean, commented JavaScript code
|
|
- ✅ Efficient force simulation (no lag)
|
|
- ✅ Proper error handling and fallbacks
|
|
- ✅ Browser compatibility (modern browsers)
|
|
- ✅ No console errors
|
|
|
|
### **Progressive Learning**
|
|
- ✅ Demonstrates new technique from web source
|
|
- ✅ Builds upon previous iterations where appropriate
|
|
- ✅ Documents what was learned and how applied
|
|
- ✅ Shows measurable improvement
|
|
- ✅ Explores new API sources
|
|
|
|
## Iteration Evolution Strategy
|
|
|
|
### **Starting Point (Iteration 1)**
|
|
- Use UN SDG API (baseline from existing project)
|
|
- Basic force-directed graph with D3.js v7
|
|
- Simple node/edge styling
|
|
- Basic hover tooltips
|
|
- Learn from: D3.js force simulation documentation
|
|
|
|
### **Early Iterations (2-5)**
|
|
- Add one new API per iteration
|
|
- Focus on core graph enhancements
|
|
- Build interactive controls
|
|
- Learn: Color scales, tooltips, filtering
|
|
|
|
### **Middle Iterations (6-12)**
|
|
- Combine multiple APIs
|
|
- Advanced interactivity
|
|
- Custom layouts and algorithms
|
|
- Learn: Graph algorithms, data merging, UX patterns
|
|
|
|
### **Advanced Iterations (13-20)**
|
|
- Real-time data integration
|
|
- Complex algorithms (community detection, centrality)
|
|
- Rich dashboards
|
|
- Learn: Advanced D3, graph theory, streaming data
|
|
|
|
### **Expert Iterations (21+)**
|
|
- AI/ML integration
|
|
- Novel visualization techniques
|
|
- Cross-platform capabilities
|
|
- Learn: Cutting-edge graph visualization research
|
|
|
|
## Success Metrics
|
|
|
|
Each iteration should improve on at least 3 of these metrics:
|
|
|
|
**Data Richness:**
|
|
- Number of API sources integrated
|
|
- Total nodes in network
|
|
- Total edges in network
|
|
- Data update frequency
|
|
|
|
**Visual Quality:**
|
|
- Color coding sophistication
|
|
- Label readability score
|
|
- Animation smoothness
|
|
- Layout clarity
|
|
|
|
**Interactivity:**
|
|
- Number of interactive features
|
|
- Response time to user actions
|
|
- Filter/search capabilities
|
|
- Information density
|
|
|
|
**Technical Innovation:**
|
|
- New techniques applied
|
|
- Performance improvements
|
|
- Code quality metrics
|
|
- Browser compatibility
|
|
|
|
## Ultra-Thinking Directive
|
|
|
|
Before each iteration, deeply consider:
|
|
|
|
**API Discovery:**
|
|
- What domain provides the most value for SDG insights?
|
|
- Which APIs have network-ready data structures?
|
|
- How can multiple APIs be meaningfully combined?
|
|
- What data relationships would be most revealing?
|
|
|
|
**Visualization Strategy:**
|
|
- What graph layout best reveals the data patterns?
|
|
- How can node/edge properties enhance understanding?
|
|
- What interactivity would provide the most insight?
|
|
- How does this build on previous iterations?
|
|
|
|
**Web Learning Application:**
|
|
- What specific technique from the web source applies here?
|
|
- How can this technique improve the visualization?
|
|
- What adaptations are needed for our use case?
|
|
- How does this advance the state of the art?
|
|
|
|
**User Value:**
|
|
- What insights does this visualization reveal?
|
|
- How intuitive is the interaction model?
|
|
- What questions can users answer with this tool?
|
|
- How does this serve SDG analysis and understanding?
|
|
|
|
**Generate visualizations that:**
|
|
- **Discover New Data**: Find and integrate novel API sources
|
|
- **Apply Web Knowledge**: Demonstrate clear learning from web research
|
|
- **Enhance Progressively**: Each iteration measurably better than the last
|
|
- **Serve SDGs**: Provide genuine value for sustainable development insights
|
|
- **Inspire Exploration**: Create compelling, informative, interactive experiences
|