Design: Offline-First Sync Engine
📖 Concept
System Design: Offline-First Data Sync Engine
This is one of the hardest mobile system design problems and a frequent Staff-level interview topic.
Requirement Clarification:
- Users can read and write data with NO internet connection
- When connectivity returns, changes sync bidirectionally
- Conflicts must be resolved without data loss
- Support for shared/collaborative data (multiple users editing)
- Sync must be efficient (delta-based, not full copies)
- Scale: 5M users, each with 10-100MB of local data
Architecture:
Client
├── Local Database (SQLite / WatermelonDB)
│ ├── Data tables (entities)
│ ├── Change log (operations journal)
│ └── Sync metadata (cursors, versions)
├── Sync Engine
│ ├── Change Tracker (detect local changes)
│ ├── Push Manager (upload changes)
│ ├── Pull Manager (download changes)
│ ├── Conflict Resolver
│ └── Queue Manager (background sync)
└── Network Monitor (connectivity status)
Server
├── Sync API (cursor-based, delta responses)
├── Conflict Resolution Service
├── Change Log (event sourcing)
└── Storage (PostgreSQL + Redis cache)
Conflict Resolution Strategies:
| Strategy | How it works | Best for |
|---|---|---|
| Last-Write-Wins (LWW) | Highest timestamp wins | Simple data, settings |
| Merge | Combine non-conflicting changes | Structured documents |
| CRDT | Mathematically conflict-free | Collaborative editing |
| User-prompted | Show both versions, user decides | Critical data |
Key Design Decisions:
- Operation-based sync (send operations, not full documents) — more efficient, enables merge
- Vector clocks for ordering events across devices — more reliable than wall clock
- Cursor-based pagination for pull sync — efficient for large datasets
- Background sync with WorkManager — reliable even if app is killed
💻 Code Example
1// === OFFLINE-FIRST SYNC ENGINE ===23// 1. Change tracking with operation log4interface SyncOperation {5 id: string;6 entityType: string; // 'note', 'task', 'contact'7 entityId: string;8 operation: 'create' | 'update' | 'delete';9 changes: Record<string, any>; // Field-level changes10 timestamp: number;11 deviceId: string;12 synced: boolean;13}1415class ChangeTracker {16 async trackChange(17 entityType: string,18 entityId: string,19 operation: 'create' | 'update' | 'delete',20 changes: Record<string, any>21 ) {22 const op: SyncOperation = {23 id: generateUUID(),24 entityType,25 entityId,26 operation,27 changes,28 timestamp: Date.now(),29 deviceId: getDeviceId(),30 synced: false,31 };3233 await localDB.insertOperation(op);3435 // Try immediate sync if online36 if (networkMonitor.isConnected) {37 syncEngine.schedulePush();38 }39 }40}4142// 2. Sync engine with conflict resolution43class SyncEngine {44 private isSyncing = false;4546 async fullSync() {47 if (this.isSyncing) return;48 this.isSyncing = true;4950 try {51 // Push first (reduces conflict window)52 await this.push();53 // Then pull54 await this.pull();55 } finally {56 this.isSyncing = false;57 }58 }5960 private async push() {61 const pendingOps = await localDB.getUnsyncedOperations();6263 if (pendingOps.length === 0) return;6465 const response = await api.pushChanges({66 deviceId: getDeviceId(),67 operations: pendingOps,68 lastSyncCursor: await localDB.getSyncCursor(),69 });7071 // Handle conflicts returned by server72 for (const conflict of response.conflicts) {73 await this.resolveConflict(conflict);74 }7576 // Mark synced operations77 await localDB.markOperationsSynced(78 pendingOps.map(op => op.id)79 );80 }8182 private async pull() {83 let cursor = await localDB.getSyncCursor();84 let hasMore = true;8586 while (hasMore) {87 const response = await api.pullChanges({88 cursor,89 limit: 100,90 deviceId: getDeviceId(), // Exclude own changes91 });9293 for (const change of response.changes) {94 await this.applyRemoteChange(change);95 }9697 cursor = response.nextCursor;98 hasMore = response.hasMore;99100 await localDB.setSyncCursor(cursor);101 }102 }103104 private async resolveConflict(conflict: ConflictInfo) {105 const local = await localDB.getEntity(conflict.entityType, conflict.entityId);106 const remote = conflict.serverValue;107108 switch (conflict.entityType) {109 case 'settings':110 // Last-write-wins for settings111 if (remote.updatedAt > local.updatedAt) {112 await localDB.upsertEntity(conflict.entityType, remote);113 }114 break;115116 case 'note':117 // Field-level merge for notes118 const merged = this.fieldLevelMerge(local, remote, conflict.baseValue);119 await localDB.upsertEntity(conflict.entityType, merged);120 break;121122 case 'transaction':123 // Never auto-resolve financial data — queue for user review124 await localDB.markConflict(conflict.entityType, conflict.entityId, {125 local,126 remote,127 });128 break;129 }130 }131132 private fieldLevelMerge(local: any, remote: any, base: any): any {133 const merged = { ...base };134135 for (const key of Object.keys(merged)) {136 const localChanged = local[key] !== base[key];137 const remoteChanged = remote[key] !== base[key];138139 if (localChanged && !remoteChanged) {140 merged[key] = local[key]; // Only local changed141 } else if (!localChanged && remoteChanged) {142 merged[key] = remote[key]; // Only remote changed143 } else if (localChanged && remoteChanged) {144 // Both changed — LWW for this field145 merged[key] = local.updatedAt > remote.updatedAt146 ? local[key]147 : remote[key];148 }149 // Neither changed — keep base value150 }151152 return merged;153 }154}155156// 3. Background sync with WorkManager (via native module)157// Schedule periodic sync when app is backgrounded158import BackgroundFetch from 'react-native-background-fetch';159160BackgroundFetch.configure({161 minimumFetchInterval: 15, // minutes162 stopOnTerminate: false,163 startOnBoot: true,164 enableHeadless: true,165}, async (taskId) => {166 await syncEngine.fullSync();167 BackgroundFetch.finish(taskId);168}, (taskId) => {169 BackgroundFetch.finish(taskId);170});
🏋️ Practice Exercise
Offline-First Design Exercises:
- Design the complete database schema for a note-taking app with sync support
- Implement a field-level merge algorithm that handles concurrent edits to different fields
- Build a sync status indicator that shows: synced, syncing, pending changes, conflict
- Design a conflict resolution UI that shows both versions and lets the user merge
- Implement cursor-based delta sync that handles pagination efficiently
- Discuss: how would you handle schema migrations when the app is offline and the server has evolved?
⚠️ Common Mistakes
Using wall clock time for conflict resolution — device clocks can be wrong; use logical clocks or server-assigned timestamps
Syncing full documents instead of deltas — wastes bandwidth and increases conflict surface area
Not handling schema evolution — if the server adds a field while the client is offline, sync breaks
Not implementing retry with backoff — failed syncs should retry with exponential backoff, not flood the server
Assuming sync is instant — design for multi-second sync on large datasets; show progress to the user
💼 Interview Questions
🎤 Mock Interview
Mock interview is powered by AI for Design: Offline-First Sync Engine. Login to unlock this feature.