canvas-website/OFFLINE_STORAGE_FEASIBILITY.md

Offline Storage Feasibility Assessment

Summary Difficulty: Medium — Feasible with Automerge's built-in support. Why it's feasible Automerge Repo supports storage adapters — you can add an IndexedDB adapter alongside the network adapter CRDT merging — Automerge handles conflict resolution automatically Existing architecture — minimal changes needed; mainly adding a storage adapter What needs to be done Add IndexedDB storage adapter (easy) Install/create an IndexedDB storage adapter Add it to the Repo configuration Fix document ID consistency (important) Currently uses repo.create() which generates new IDs each time Need to use repo.find(roomId) or store a roomId → documentId mapping This is the main blocker for local storage persistence Handle online/offline transitions (medium) Listen for network status changes Trigger sync when coming back online The existing network adapter already handles reconnection Update initialization logic (medium) Load from IndexedDB first (instant offline access) Then sync with server when online Automerge will automatically merge changes Estimated effort Research & setup: 1-2 hours Implementation: 4-6 hours Testing: 2-3 hours Total: ~1 day of focused work Main challenge The document ID consistency issue. The code comment says "We can't use repo.find() with a custom ID because Automerge requires specific document ID formats" — this needs to be resolved. Options: Store a mapping of roomId → documentId in IndexedDB Use a deterministic algorithm to convert roomId to a valid Automerge document ID Use Automerge's document ID generation but persist the mapping Benefits Instant offline access — no loading spinners Automatic conflict resolution — Automerge handles merging Better UX — works seamlessly offline Resilience — works even if server is temporarily unavailable I've created a detailed feasibility document at OFFLINE_STORAGE_FEASIBILITY.md with implementation steps, code examples, and considerations. Should I start implementing this, or do you want to review the feasibility document first?

Summary

Difficulty: Medium - The implementation is straightforward thanks to Automerge's built-in support for storage adapters, but requires careful integration with the existing sync architecture.

Current Architecture

1. Client-side: Uses @automerge/automerge-repo with CloudflareNetworkAdapter for WebSocket sync
2. Server-side: AutomergeDurableObject stores documents in R2 and handles WebSocket connections
3. Persistence flow:
   • Client saves to worker via POST /room/:roomId
   • Worker persists to R2 (throttled to every 2 seconds)
   • Client loads initial data from server via GET /room/:roomId

What's Needed

1. Add IndexedDB Storage Adapter (Easy)

Automerge Repo supports storage adapters out of the box. You'll need to:

• Install @automerge/automerge-repo-storage-indexeddb (if available) or create a custom IndexedDB adapter
• Add the storage adapter to the Repo configuration alongside the network adapter
• The Repo will automatically persist document changes to IndexedDB

Code changes needed:

// In useAutomergeSyncRepo.ts
import { IndexedDBStorageAdapter } from "@automerge/automerge-repo-storage-indexeddb"

const [repo] = useState(() => {
  const adapter = new CloudflareNetworkAdapter(workerUrl, roomId, applyJsonSyncData)
  const storageAdapter = new IndexedDBStorageAdapter() // Add this
  return new Repo({
    network: [adapter],
    storage: [storageAdapter] // Add this
  })
})

2. Load from Local Storage on Startup (Medium)

Modify the initialization logic to:

• Check IndexedDB for existing document data
• Load from IndexedDB first (for instant offline access)
• Then sync with server when online
• Automerge will automatically merge local and remote changes

Code changes needed:

// In useAutomergeSyncRepo.ts - modify initializeHandle
const initializeHandle = async () => {
  // Check if document exists in IndexedDB first
  const localDoc = await repo.find(roomId) // This will load from IndexedDB if available
  
  // Then sync with server (if online)
  if (navigator.onLine) {
    // Existing server sync logic
  }
}

3. Handle Online/Offline Transitions (Medium)

• Detect network status changes
• When coming online, ensure sync happens
• The existing CloudflareNetworkAdapter already handles reconnection, but you may want to add explicit sync triggers

Code changes needed:

// Add network status listener
useEffect(() => {
  const handleOnline = () => {
    console.log('🌐 Back online - syncing with server')
    // Trigger sync - Automerge will handle merging automatically
    if (handle) {
      // The network adapter will automatically reconnect and sync
    }
  }
  
  window.addEventListener('online', handleOnline)
  return () => window.removeEventListener('online', handleOnline)
}, [handle])

4. Document ID Consistency (Important)

Currently, the code creates a new document handle each time (repo.create()). For local storage to work properly, you need:

• Consistent document IDs per room
• The challenge: Automerge requires specific document ID formats (like automerge:xxxxx)
• Solution options:
  1. Use repo.find() with a properly formatted Automerge document ID (derive from roomId)
  2. Store a mapping of roomId → documentId in IndexedDB
  3. Use a deterministic way to generate document IDs from roomId

Code changes needed:

// Option 1: Generate deterministic Automerge document ID from roomId
const documentId = `automerge:${roomId}` // May need proper formatting
const handle = repo.find(documentId) // This will load from IndexedDB or create new

// Option 2: Store mapping in IndexedDB
const storedMapping = await getDocumentIdMapping(roomId)
const documentId = storedMapping || generateNewDocumentId()
const handle = repo.find(documentId)
await saveDocumentIdMapping(roomId, documentId)

Note: The current code comment says "We can't use repo.find() with a custom ID because Automerge requires specific document ID formats" - this needs to be resolved. You may need to:

• Use Automerge's document ID generation but store the mapping
• Or use a deterministic algorithm to convert roomId to valid Automerge document ID format

Benefits

1. Instant Offline Access: Users can immediately see and edit their data without waiting for server response
2. Automatic Merging: Automerge's CRDT nature means local and remote changes merge automatically without conflicts
3. Better UX: No loading spinners when offline - data is instantly available
4. Resilience: Works even if server is temporarily unavailable

Challenges & Considerations

1. Storage Quota Limits

• IndexedDB has browser-specific limits (typically 50% of disk space)
• Large documents could hit quota limits
• Solution: Monitor storage usage and implement cleanup for old documents

2. Document ID Management

• Need to ensure consistent document IDs per room
• Current code uses repo.create() which generates new IDs
• Solution: Use repo.find(roomId) with a consistent ID format

3. Initial Load Strategy

• Should load from IndexedDB first (fast) or server first (fresh)?
• Recommendation: Load from IndexedDB first for instant UI, then sync with server in background

4. Conflict Resolution

• Automerge handles this automatically, but you may want to show users when their offline changes were merged
• Solution: Use Automerge's change tracking to show merge notifications

5. Storage Adapter Availability

• Need to verify if @automerge/automerge-repo-storage-indexeddb exists
• If not, you'll need to create a custom adapter (still straightforward)

Implementation Steps

1. Research: Check if @automerge/automerge-repo-storage-indexeddb package exists
2. Install: Add storage adapter package or create custom adapter
3. Modify Repo Setup: Add storage adapter to Repo configuration
4. Update Document Loading: Use repo.find() instead of repo.create() for consistent IDs
5. Add Network Detection: Listen for online/offline events
6. Test: Verify offline editing works and syncs correctly when back online
7. Handle Edge Cases: Storage quota, document size limits, etc.

Estimated Effort

• Research & Setup: 1-2 hours
• Implementation: 4-6 hours
• Testing: 2-3 hours
• Total: ~1 day of focused work

Code Locations to Modify

1. src/automerge/useAutomergeSyncRepo.ts - Main sync hook (add storage adapter, modify initialization)
2. src/automerge/CloudflareAdapter.ts - Network adapter (may need minor changes for offline detection)
3. Potentially create: src/automerge/IndexedDBStorageAdapter.ts - If custom adapter needed

Conclusion

This is a medium-complexity feature that's very feasible. Automerge's architecture is designed for this exact use case, and the main work is:

1. Adding the storage adapter (straightforward)
2. Ensuring consistent document IDs (important fix)
3. Handling online/offline transitions (moderate complexity)

The biggest benefit is that Automerge's CRDT nature means you don't need to write complex merge logic - it handles conflict resolution automatically.

---

Related: Google Data Sovereignty

Beyond canvas document storage, we also support importing and securely storing Google Workspace data locally. See docs/GOOGLE_DATA_SOVEREIGNTY.md for the complete architecture covering:

• Gmail - Import and encrypt emails locally
• Drive - Import and encrypt documents locally
• Photos - Import thumbnails with on-demand full resolution
• Calendar - Import and encrypt events locally

Key principles:

1. Local-first: All data stored in encrypted IndexedDB
2. User-controlled encryption: Keys derived from WebCrypto auth, never leave browser
3. Selective sharing: Choose what to share to canvas boards
4. Optional R2 backup: Encrypted cloud backup (you hold the keys)

This builds on the same IndexedDB + Automerge foundation described above.