canvas-website/test-r2-conversion.md

Testing R2 Data Conversion

This guide helps you test the data conversion from old tldraw sync format to new automerge sync format using actual data from your R2 bucket.

Overview

The conversion system handles three data formats:

1. Automerge Array Format: [{ state: {...} }, ...]
2. Store Format: { store: { "recordId": {...}, ... }, schema: {...} } (already converted)
3. Old Documents Format: { documents: [{ state: {...} }, ...] } (legacy tldraw sync)

Testing Steps

1. Identify Test Rooms

First, identify rooms in your R2 bucket that use the old format:

# List all rooms in R2
# You can use wrangler CLI or Cloudflare dashboard
wrangler r2 object list TLDRAW_BUCKET --prefix "rooms/"

2. Check Data Format

For each room, check its format:

// Example: Check a room's format
const roomId = "your-room-id"
const doc = await r2.get(`rooms/${roomId}`)
const data = await doc.json()

// Check format
if (Array.isArray(data)) {
  console.log("Format: Automerge Array")
} else if (data.store) {
  console.log("Format: Store Format (already converted)")
} else if (data.documents) {
  console.log("Format: Old Documents Format (needs conversion)")
} else {
  console.log("Format: Unknown")
}

3. Test Conversion

The conversion happens automatically when a room is loaded. To test:

1. Load the room in your app - The AutomergeDurableObject.getDocument() method will automatically detect and convert the format
2. Check the logs - Look for conversion statistics in the worker logs:
   • 📊 Automerge to Store conversion statistics
   • 📊 Documents to Store migration statistics
   • 📊 Shape migration statistics

4. Verify Data Integrity

After conversion, verify:

1. All shapes are present: Check that shape count matches
2. Custom shapes preserved: Verify ObsNote, Holon, etc. have all their properties
3. Custom records preserved: Check that obsidian_vault records are present
4. No validation errors: Shapes should render without errors

Expected Log Output

When a room is converted, you should see logs like:

Converting Automerge document format to store format for room abc123
📊 Automerge to Store conversion statistics: {
  total: 150,
  converted: 148,
  skipped: 2,
  errors: 0,
  storeKeys: 148,
  customRecordCount: 1,
  customRecordIds: ['obsidian_vault:test'],
  errorCount: 0
}
✅ Verified 1 custom records preserved during conversion

🔄 Server-side: Starting shape migration for room abc123
📊 Shape migration statistics: {
  total: 120,
  migrated: 45,
  skipped: 75,
  errors: 0,
  shapeTypes: { geo: 50, arrow: 20, ObsNote: 10, ... },
  customShapesCount: 10,
  customShapeIds: ['shape:obs1', 'shape:holon1', ...],
  errorCount: 0
}
✅ Verified 10 custom shapes preserved during migration

Manual Testing Script

You can create a test script to verify conversion:

// test-r2-room.ts
import { AutomergeDurableObject } from './worker/AutomergeDurableObject'

async function testRoomConversion(roomId: string) {
  // This would need to be run in a Cloudflare Worker context
  // or use wrangler dev to test locally
  
  const env = {
    TLDRAW_BUCKET: yourR2Bucket
  }
  
  // Create a mock Durable Object state
  const ctx = {
    storage: {
      get: async (key: string) => roomId,
      put: async (key: string, value: any) => {}
    },
    blockConcurrencyWhile: async (fn: () => Promise<void>) => await fn()
  }
  
  const do = new AutomergeDurableObject(ctx as any, env as any)
  
  // Load and convert
  const doc = await do.getDocument()
  
  // Verify
  console.log('Conversion complete:', {
    storeKeys: Object.keys(doc.store).length,
    shapes: Object.values(doc.store).filter((r: any) => r.typeName === 'shape').length,
    customRecords: Object.values(doc.store).filter((r: any) => 
      r.id && typeof r.id === 'string' && r.id.startsWith('obsidian_vault:')
    ).length
  })
}

Common Issues and Solutions

Issue: Records are skipped during conversion

Cause: Missing required fields (id, typeName, state)

Solution: Check the error details in logs. The conversion will skip invalid records but log warnings.

Issue: Custom shapes missing properties

Cause: Shape migration may have failed

Solution: Check shape migration logs. Custom shape props should be preserved automatically.

Issue: Custom records (obsidian_vault) missing

Cause: They were filtered out during conversion

Solution: This shouldn't happen - custom records are preserved. Check logs for customRecordCount.

Validation Checklist

After conversion, verify:

• All shapes are present (count matches)
• Custom shapes (ObsNote, Holon, etc.) have all properties
• Custom records (obsidian_vault) are preserved
• No validation errors when loading the room
• Shapes render correctly in the UI
• All text content is preserved
• All metadata is preserved

Rollback Plan

If conversion fails:

1. The original data in R2 is not modified until the first save
2. You can restore from backup if needed
3. Check worker logs for specific errors
4. The conversion creates a new document if it fails, so original data is safe

Next Steps

1. Test with a few sample rooms first
2. Monitor logs for any warnings or errors
3. Verify data integrity after conversion
4. Once confident, the conversion will happen automatically for all rooms