Tag: Write-Ahead Logging (WAL)

Documentation tagged with Write-Ahead Logging (WAL) in the Geode graph database. WAL is a fundamental technique for ensuring data durability and enabling crash recovery—every database modification is recorded in a sequential log file before being applied to the database itself. <h3 id="introduction-to-write-ahead-logging" class="position-relative d-flex align-items-center group"> Introduction to Write-Ahead Logging <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="introduction-to-write-ahead-logging" aria-haspopup="dialog" aria-label="Share link: Introduction to Write-Ahead Logging"> Share link </button> </h3><div id="headingShareModal" class="heading-share-modal" role="dialog" aria-modal="true" aria-labelledby="headingShareTitle" hidden> <div class="hsm-dialog" role="document"> <div class="hsm-header"> <h2 id="headingShareTitle" class="h6 mb-0 fw-bold">Share this section</h2> <button type="button" class="hsm-close" aria-label="Close"> </button> </div> <div class="hsm-body"> <label for="headingShareInput" class="form-label small text-muted mb-1 text-uppercase fw-bold" style="font-size: 0.7rem; letter-spacing: 0.5px;">Permalink</label> <div class="input-group mb-4 hsm-url-group"> <input id="headingShareInput" type="text" class="form-control font-monospace" readonly aria-readonly="true" style="font-size: 0.85rem;" /> <button class="btn btn-primary hsm-copy" type="button" aria-label="Copy" title="Copy"> </button> </div> <div class="small fw-bold mb-2 text-muted text-uppercase" style="font-size: 0.7rem; letter-spacing: 0.5px;">Share via</div> <div class="hsm-share-grid"> <a id="share-twitter" class="btn btn-outline-secondary w-100" target="_blank" rel="noopener noreferrer"> Twitter </a> <a id="share-linkedin" class="btn btn-outline-secondary w-100" target="_blank" rel="noopener noreferrer"> LinkedIn </a> <a id="share-facebook" class="btn btn-outline-secondary w-100" target="_blank" rel="noopener noreferrer"> Facebook </a> </div> </div> </div> </div> <style> .heading-share-modal { position: fixed; inset: 0; display: flex; justify-content: center; align-items: center; background: rgba(0, 0, 0, 0.6); z-index: 1050; padding: 1rem; backdrop-filter: blur(4px); -webkit-backdrop-filter: blur(4px); } .heading-share-modal[hidden] { display: none !important; } .hsm-dialog { max-width: 420px; width: 100%; background: var(--bs-body-bg, #fff); color: var(--bs-body-color, #212529); border: 1px solid var(--bs-border-color, rgba(0,0,0,0.1)); border-radius: 1rem; box-shadow: 0 25px 50px -12px rgba(0, 0, 0, 0.25); overflow: hidden; animation: hsm-fade-in 0.2s ease-out; } @keyframes hsm-fade-in { from { opacity: 0; transform: scale(0.95); } to { opacity: 1; transform: scale(1); } } [data-bs-theme="dark"] .hsm-dialog { background: #1e293b; border-color: rgba(255,255,255,0.1); color: #f8f9fa; } .hsm-header { display: flex; justify-content: space-between; align-items: center; padding: 1rem 1.5rem; border-bottom: 1px solid var(--bs-border-color, rgba(0,0,0,0.1)); background: rgba(0,0,0,0.02); } [data-bs-theme="dark"] .hsm-header { background: rgba(255,255,255,0.02); border-color: rgba(255,255,255,0.1); } .hsm-close { background: transparent; border: none; color: inherit; opacity: 0.5; padding: 0.25rem 0.5rem; border-radius: 0.25rem; font-size: 1.2rem; line-height: 1; transition: opacity 0.2s; } .hsm-close:hover { opacity: 1; } .hsm-body { padding: 1.5rem; } .hsm-url-group { display: flex !important; align-items: stretch; } .hsm-url-group .form-control { flex: 1; min-width: 0; margin: 0; background: var(--bs-secondary-bg, #f8f9fa); border-color: var(--bs-border-color, #dee2e6); border-top-right-radius: 0; border-bottom-right-radius: 0; height: 42px; } .hsm-url-group .btn { flex: 0 0 auto; margin: 0; margin-left: -1px; border-top-left-radius: 0; border-bottom-left-radius: 0; height: 42px; display: flex; align-items: center; justify-content: center; padding: 0 1.25rem; z-index: 2; } [data-bs-theme="dark"] .hsm-url-group .form-control { background: #0f172a; border-color: #334155; color: #e2e8f0; } .hsm-share-grid { display: flex; flex-direction: column; gap: 0.5rem; } .hsm-share-grid .btn { display: flex; align-items: center; justify-content: center; font-size: 0.9rem; padding: 0.6rem; border-color: var(--bs-border-color); width: 100%; } [data-bs-theme="dark"] .hsm-share-grid .btn { color: #e2e8f0; border-color: #475569; } [data-bs-theme="dark"] .hsm-share-grid .btn:hover { background: #334155; border-color: #cbd5e1; } </style> <script> (function(){ const modal = document.getElementById('headingShareModal'); if(!modal) return; const input = modal.querySelector('#headingShareInput'); const copyBtn = modal.querySelector('.hsm-copy'); const twitter = modal.querySelector('#share-twitter'); const linkedin = modal.querySelector('#share-linkedin'); const facebook = modal.querySelector('#share-facebook'); const closeBtn = modal.querySelector('.hsm-close'); let lastFocus=null; let trapBound=false; function buildUrl(id){ return window.location.origin + window.location.pathname + '#' + id; } function isOpen(){ return !modal.hasAttribute('hidden'); } function hydrate(id){ const url=buildUrl(id); input.value=url; const enc=encodeURIComponent(url); const text=encodeURIComponent(document.title); if(twitter) twitter.href=`https://twitter.com/intent/tweet?url=${enc}&text=${text}`; if(linkedin) linkedin.href=`https://www.linkedin.com/sharing/share-offsite/?url=${enc}`; if(facebook) facebook.href=`https://www.facebook.com/sharer/sharer.php?u=${enc}`; } function openModal(id){ lastFocus=document.activeElement; hydrate(id); if(!isOpen()){ modal.removeAttribute('hidden'); } requestAnimationFrame(()=>{ input.focus(); }); trapFocus(); } function closeModal(){ if(!isOpen()) return; modal.setAttribute('hidden',''); if(lastFocus && typeof lastFocus.focus==='function') lastFocus.focus(); } function copyCurrent(){ try{ navigator.clipboard.writeText(input.value).then(()=>feedback(true),()=>fallback()); } catch(e){ fallback(); } } function fallback(){ input.select(); try{ document.execCommand('copy'); feedback(true);}catch(e){ feedback(false);} } function feedback(ok){ if(!copyBtn) return; const icon=copyBtn.querySelector('i'); if(!icon) return; const prev=copyBtn.getAttribute('data-prev')||icon.className; if(!copyBtn.getAttribute('data-prev')) copyBtn.setAttribute('data-prev',prev); icon.className= ok ? 'fa-duotone fa-clipboard-check':'fa-duotone fa-circle-exclamation'; setTimeout(()=>{ icon.className=prev; },1800); } function handleShareClick(e){ e.preventDefault(); const btn=e.currentTarget; const id=btn.getAttribute('data-share-target'); if(id) openModal(id); } function bindShareButtons(){ document.querySelectorAll('.h-share').forEach(btn=>{ if(!btn.dataset.hShareBound){ btn.addEventListener('click', handleShareClick); btn.dataset.hShareBound='1'; } }); } bindShareButtons(); if(document.readyState==='loading'){ document.addEventListener('DOMContentLoaded', bindShareButtons); } else { requestAnimationFrame(bindShareButtons); } document.addEventListener('click', function(e){ const shareBtn=e.target.closest && e.target.closest('.h-share'); if(shareBtn && !shareBtn.dataset.hShareBound){ handleShareClick.call(shareBtn, e); } }, true); document.addEventListener('click', e=>{ if(e.target===modal) closeModal(); if(e.target.closest && e.target.closest('.hsm-close')){ e.preventDefault(); closeModal(); } if(copyBtn && (e.target===copyBtn || (e.target.closest && e.target.closest('.hsm-copy')))) { e.preventDefault(); copyCurrent(); } }); document.addEventListener('keydown', e=>{ if(e.key==='Escape' && isOpen()) closeModal(); }); function trapFocus(){ if(trapBound) return; trapBound=true; modal.addEventListener('keydown', f=>{ if(f.key==='Tab' && isOpen()){ const focusable=[...modal.querySelectorAll('a[href],button,input,textarea,select,[tabindex]:not([tabindex="-1"])')].filter(el=>!el.hasAttribute('disabled')); if(!focusable.length) return; const first=focusable[0]; const last=focusable[focusable.length-1]; if(f.shiftKey && document.activeElement===first){ f.preventDefault(); last.focus(); } else if(!f.shiftKey && document.activeElement===last){ f.preventDefault(); first.focus(); } } }); } if(closeBtn) closeBtn.addEventListener('click', e=>{ e.preventDefault(); closeModal(); }); })(); </script>Write-Ahead Logging (WAL) is the cornerstone of database durability. The principle is elegantly simple: before modifying any data in the database, first write a description of the change to a sequential log file on persistent storage. This ensures that even if the system crashes mid-operation, the database can recover to a consistent state by replaying the log. WAL has been a standard component of database systems since the 1970s, used by virtually every production database including PostgreSQL, Oracle, MySQL, and now Geode. The technique solves a fundamental problem: random writes to database pages are expensive (requiring disk seeks and scattered I/O), but sequential writes to a log file are fast (leveraging sequential disk throughput). By committing transactions via WAL, Geode achieves both: <ul> <li>Fast commits: Write sequential log records instead of scattered database pages</li> <li>Crash recovery: Replay the WAL to reconstruct lost in-memory state</li> <li>Point-in-time recovery: Restore database to any historical moment</li> <li>Replication: Ship WAL records to replicas for synchronization</li> </ul> Geode’s WAL implementation provides full ACID durability guarantees while maintaining high write throughput—critical for production graph workloads. <h3 id="core-wal-concepts" class="position-relative d-flex align-items-center group"> Core WAL Concepts <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="core-wal-concepts" aria-haspopup="dialog" aria-label="Share link: Core WAL Concepts"> Share link </button> </h3> <h4 id="the-wal-protocol" class="position-relative d-flex align-items-center group"> The WAL Protocol <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="the-wal-protocol" aria-haspopup="dialog" aria-label="Share link: The WAL Protocol"> Share link </button> </h4>The fundamental WAL protocol consists of three rules: <ol> <li>Write-Ahead Rule: Log records must reach persistent storage before data pages</li> <li>Force-Log-at-Commit: All log records for a transaction must be durable before commit returns</li> <li>No-Force Policy: Data pages may remain in memory; only WAL must be forced to disk</li> </ol> These rules ensure that: <ul> <li>Committed transactions survive crashes (durability)</li> <li>Uncommitted transactions can be rolled back (atomicity)</li> <li>The database can be reconstructed from the WAL (recoverability)</li> </ul> <h4 id="log-sequence-numbers-lsn" class="position-relative d-flex align-items-center group"> Log Sequence Numbers (LSN) <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="log-sequence-numbers-lsn" aria-haspopup="dialog" aria-label="Share link: Log Sequence Numbers (LSN)"> Share link </button> </h4>Every WAL record is assigned a monotonically increasing Log Sequence Number (LSN). LSNs serve multiple purposes: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback">LSN: 64-bit integer representing log position Example: 0x0000000123456789 Components: - High 32 bits: WAL file number - Low 32 bits: Offset within file </code></pre></div>LSNs enable: <ul> <li>Ordering of operations</li> <li>Tracking which records have been applied</li> <li>Identifying the recovery start point</li> <li>Coordinating replication</li> </ul> <h4 id="wal-record-structure" class="position-relative d-flex align-items-center group"> WAL Record Structure <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="wal-record-structure" aria-haspopup="dialog" aria-label="Share link: WAL Record Structure"> Share link </button> </h4>Each WAL record contains: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback">WAL Record Format: [LSN][Transaction ID][Operation Type][Before Image][After Image][Checksum] Example: [0x123456][tx-98765][UPDATE_NODE][{name: "Alice"}][{name: "Alice", age: 30}][crc32: 0xABCD] </code></pre></div>Components: <ul> <li>LSN: Unique identifier for this record</li> <li>Transaction ID: Which transaction generated this operation</li> <li>Operation Type: INSERT, UPDATE, DELETE, BEGIN, COMMIT, ABORT</li> <li>Before Image: Data state before modification (for undo)</li> <li>After Image: Data state after modification (for redo)</li> <li>Checksum: Integrity verification</li> </ul> <h4 id="checkpointing" class="position-relative d-flex align-items-center group"> Checkpointing <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="checkpointing" aria-haspopup="dialog" aria-label="Share link: Checkpointing"> Share link </button> </h4>Periodically, Geode performs a checkpoint: <ol> <li>Flush all dirty pages to disk</li> <li>Write a checkpoint record to WAL</li> <li>Record the LSN in a known location</li> </ol> During recovery, Geode only needs to replay WAL records after the last checkpoint, dramatically reducing recovery time. <h3 id="how-wal-works-in-geode" class="position-relative d-flex align-items-center group"> How WAL Works in Geode <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="how-wal-works-in-geode" aria-haspopup="dialog" aria-label="Share link: How WAL Works in Geode"> Share link </button> </h3> <h4 id="write-path" class="position-relative d-flex align-items-center group"> Write Path <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="write-path" aria-haspopup="dialog" aria-label="Share link: Write Path"> Share link </button> </h4>When a transaction modifies data: <ol> <li>Generate WAL Record: Create log record describing the change</li> <li>Append to WAL Buffer: Write to in-memory buffer (fast)</li> <li>Modify In-Memory Data: Update in-memory database structures</li> <li>Flush WAL on Commit: Call <code>fsync()</code> to ensure WAL reaches disk</li> <li>Return Success: Commit completes only after WAL is durable</li> </ol> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-gql" data-lang="gql">BEGIN TRANSACTION; MATCH (p:Person {id: 123}) SET p.age = 30; -- At this point, WAL record exists in memory buffer COMMIT; -- fsync() forces WAL to disk, then commit returns </code></pre></div> <h4 id="crash-recovery" class="position-relative d-flex align-items-center group"> Crash Recovery <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="crash-recovery" aria-haspopup="dialog" aria-label="Share link: Crash Recovery"> Share link </button> </h4>On startup after a crash, Geode performs WAL replay: <ol> <li>Find Last Checkpoint: Read checkpoint LSN from control file</li> <li>Read WAL: Start reading from checkpoint LSN</li> <li>Identify Transactions: Separate committed from uncommitted transactions</li> <li>Redo Pass: Replay committed transactions to reconstruct state</li> <li>Undo Pass: Roll back uncommitted transactions</li> <li>Resume Operation: Database is now consistent and ready</li> </ol> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback">Recovery Example: Checkpoint at LSN 1000 Crash at LSN 1500 WAL Records: [1001][tx-1][BEGIN] [1002][tx-1][INSERT Node(Person, id=1)] [1003][tx-1][COMMIT] [1004][tx-2][BEGIN] [1005][tx-2][INSERT Node(Person, id=2)] [CRASH - no commit for tx-2] Recovery: - Replay tx-1: INSERT Node(Person, id=1) - COMMITTED - Rollback tx-2: Undo INSERT Node(Person, id=2) - UNCOMMITTED </code></pre></div> <h4 id="wal-archival-and-rotation" class="position-relative d-flex align-items-center group"> WAL Archival and Rotation <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="wal-archival-and-rotation" aria-haspopup="dialog" aria-label="Share link: WAL Archival and Rotation"> Share link </button> </h4>WAL files grow continuously. Geode manages this through: WAL Rotation: When a WAL file reaches a size limit (default 64MB), start a new file <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback">wal/ ├── 00000001.wal (complete, archived) ├── 00000002.wal (complete, archived) ├── 00000003.wal (complete, archived) └── 00000004.wal (current, active) </code></pre></div>WAL Archival: Old WAL files are moved to archive storage for: <ul> <li>Point-in-time recovery</li> <li>Replication to new replicas</li> <li>Audit and compliance</li> </ul> WAL Recycling: After checkpoint, archived WAL files older than the retention period are deleted or recycled. <h4 id="integration-with-mvcc" class="position-relative d-flex align-items-center group"> Integration with MVCC <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="integration-with-mvcc" aria-haspopup="dialog" aria-label="Share link: Integration with MVCC"> Share link </button> </h4>WAL works hand-in-hand with Multi-Version Concurrency Control: <ul> <li>Before Images: WAL records contain old version data for rollback</li> <li>After Images: WAL records contain new version data for redo</li> <li>Version Chains: WAL enables reconstruction of version chains after crash</li> <li>Garbage Collection: WAL archival coordinated with MVCC GC</li> </ul> <h3 id="use-cases-and-benefits" class="position-relative d-flex align-items-center group"> Use Cases and Benefits <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="use-cases-and-benefits" aria-haspopup="dialog" aria-label="Share link: Use Cases and Benefits"> Share link </button> </h3> <h4 id="crash-recovery-1" class="position-relative d-flex align-items-center group"> Crash Recovery <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="crash-recovery-1" aria-haspopup="dialog" aria-label="Share link: Crash Recovery"> Share link </button> </h4>WAL enables automatic recovery from crashes: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"># Server crashes during transaction $ geode serve [CRASH - power failure] # Restart automatically recovers $ geode serve [2026-01-24 10:00:00] Starting crash recovery... [2026-01-24 10:00:01] Reading WAL from LSN 123456789 [2026-01-24 10:00:02] Replaying 1523 committed transactions [2026-01-24 10:00:03] Rolling back 2 uncommitted transactions [2026-01-24 10:00:04] Recovery complete [2026-01-24 10:00:05] Database ready </code></pre></div> <h4 id="point-in-time-recovery" class="position-relative d-flex align-items-center group"> Point-in-Time Recovery <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="point-in-time-recovery" aria-haspopup="dialog" aria-label="Share link: Point-in-Time Recovery"> Share link </button> </h4>WAL enables time-travel recovery: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"># Restore database to specific timestamp $ geode restore --time "2026-01-24T09:00:00Z" [2026-01-24 10:05:00] Restoring from checkpoint at 08:00:00 [2026-01-24 10:05:01] Replaying WAL up to 09:00:00 [2026-01-24 10:05:02] Restored to 2026-01-24T09:00:00Z </code></pre></div>This is invaluable for: <ul> <li>Recovering from accidental data deletion</li> <li>Investigating historical states for debugging</li> <li>Compliance and audit requirements</li> </ul> <h4 id="replication" class="position-relative d-flex align-items-center group"> Replication <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="replication" aria-haspopup="dialog" aria-label="Share link: Replication"> Share link </button> </h4>WAL enables efficient replication: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback">Primary Server Replica Server | | |----[WAL Record 1001]---------->| |----[WAL Record 1002]---------->| |----[WAL Record 1003]---------->| | | | [Apply WAL] </code></pre></div>Replicas apply WAL records to stay synchronized with the primary, enabling: <ul> <li>High availability (fail over to replica)</li> <li>Read scalability (distribute reads across replicas)</li> <li>Geographic distribution (replicas in multiple regions)</li> </ul> <h4 id="audit-logging" class="position-relative d-flex align-items-center group"> Audit Logging <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="audit-logging" aria-haspopup="dialog" aria-label="Share link: Audit Logging"> Share link </button> </h4>WAL provides a complete audit trail: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-gql" data-lang="gql">-- Query WAL for specific transaction CALL dbms.audit.wal.query({ transaction_id: "tx-12345", time_range: {start: "2026-01-24T00:00:00Z", end: "2026-01-24T23:59:59Z"} }) YIELD lsn, operation, before_image, after_image, timestamp RETURN lsn, operation, before_image, after_image, timestamp; </code></pre></div>This supports: <ul> <li>Compliance (SOC 2, HIPAA, GDPR)</li> <li>Forensic investigation</li> <li>Change tracking</li> </ul> <h3 id="best-practices" class="position-relative d-flex align-items-center group"> Best Practices <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="best-practices" aria-haspopup="dialog" aria-label="Share link: Best Practices"> Share link </button> </h3> <h4 id="wal-storage-configuration" class="position-relative d-flex align-items-center group"> WAL Storage Configuration <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="wal-storage-configuration" aria-haspopup="dialog" aria-label="Share link: WAL Storage Configuration"> Share link </button> </h4><ol> <li> Use dedicated storage: Place WAL on separate disks from data files <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash">$ geode serve --wal-dir /mnt/wal-ssd --data-dir /mnt/data-hdd </code></pre></div></li> <li> Use SSDs for WAL: Sequential writes benefit from SSD latency <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback">WAL on HDD: ~10ms fsync latency → 100 commits/sec WAL on SSD: ~0.1ms fsync latency → 10,000 commits/sec </code></pre></div></li> <li> RAID configuration: Use RAID 1 or RAID 10 for WAL reliability </li> </ol> <h4 id="checkpoint-tuning" class="position-relative d-flex align-items-center group"> Checkpoint Tuning <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="checkpoint-tuning" aria-haspopup="dialog" aria-label="Share link: Checkpoint Tuning"> Share link </button> </h4>Configure checkpoint frequency to balance: <ul> <li>Recovery time: More frequent checkpoints → faster recovery</li> <li>Performance: Less frequent checkpoints → less I/O overhead</li> </ul> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"># geode.yaml wal: checkpoint_interval: 5m # Checkpoint every 5 minutes checkpoint_timeout: 30s # Max time for checkpoint max_wal_size: 1GB # Force checkpoint if WAL exceeds 1GB </code></pre></div> <h4 id="wal-archival-strategy" class="position-relative d-flex align-items-center group"> WAL Archival Strategy <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="wal-archival-strategy" aria-haspopup="dialog" aria-label="Share link: WAL Archival Strategy"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"># geode.yaml wal: archive_mode: enabled archive_command: "cp %p /mnt/wal-archive/%f" archive_timeout: 60s retention_days: 30 </code></pre></div>Configure archival based on requirements: <ul> <li>Compliance: Long retention (90+ days)</li> <li>Replication: Keep WAL until all replicas apply it</li> <li>Storage cost: Balance retention with storage costs</li> </ul> <h4 id="monitoring-wal-health" class="position-relative d-flex align-items-center group"> Monitoring WAL Health <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="monitoring-wal-health" aria-haspopup="dialog" aria-label="Share link: Monitoring WAL Health"> Share link </button> </h4>Track critical WAL metrics: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-gql" data-lang="gql">-- WAL metrics CALL dbms.monitor.wal.stats() YIELD current_lsn, checkpoint_lsn, wal_files, wal_size, flush_rate; -- WAL lag (replication) CALL dbms.monitor.replication.lag() YIELD replica, wal_lag_bytes, wal_lag_seconds; -- Checkpoint statistics CALL dbms.monitor.checkpoint.stats() YIELD last_checkpoint, duration, pages_written; </code></pre></div>Alert on: <ul> <li>WAL accumulation: WAL files growing without bound</li> <li>Slow fsync: Commit latency increasing</li> <li>Replication lag: Replicas falling behind</li> </ul> <h3 id="troubleshooting" class="position-relative d-flex align-items-center group"> Troubleshooting <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="troubleshooting" aria-haspopup="dialog" aria-label="Share link: Troubleshooting"> Share link </button> </h3> <h4 id="wal-disk-full" class="position-relative d-flex align-items-center group"> WAL Disk Full <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="wal-disk-full" aria-haspopup="dialog" aria-label="Share link: WAL Disk Full"> Share link </button> </h4>Symptom: Commits fail with “WAL disk full” error Cause: WAL directory ran out of space Solution: <ol> <li>Add storage to WAL directory</li> <li>Reduce WAL retention</li> <li>Increase checkpoint frequency</li> <li>Archive old WAL files to cheaper storage</li> </ol> <h4 id="slow-commits" class="position-relative d-flex align-items-center group"> Slow Commits <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="slow-commits" aria-haspopup="dialog" aria-label="Share link: Slow Commits"> Share link </button> </h4>Symptom: High commit latency Cause: Slow fsync() calls Solution: <ol> <li>Move WAL to faster storage (SSD)</li> <li>Check disk I/O contention</li> <li>Tune filesystem mount options (noatime, data=writeback)</li> <li>Consider group commit optimization</li> </ol> <h4 id="long-recovery-times" class="position-relative d-flex align-items-center group"> Long Recovery Times <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="long-recovery-times" aria-haspopup="dialog" aria-label="Share link: Long Recovery Times"> Share link </button> </h4>Symptom: Database takes minutes to recover after crash Cause: Large amount of WAL to replay Solution: <ol> <li>Increase checkpoint frequency</li> <li>Reduce transaction size</li> <li>Add more memory for in-memory buffers</li> <li>Use parallel WAL replay (if available)</li> </ol> <h4 id="corrupted-wal" class="position-relative d-flex align-items-center group"> Corrupted WAL <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="corrupted-wal" aria-haspopup="dialog" aria-label="Share link: Corrupted WAL"> Share link </button> </h4>Symptom: Recovery fails with checksum mismatch Cause: Disk corruption, partial write, bit flip Solution: <ol> <li>Restore from backup</li> <li>Recover to last valid LSN (may lose recent commits)</li> <li>Use WAL archive if available</li> <li>Check hardware (bad disk, bad memory)</li> </ol> <h3 id="performance-considerations" class="position-relative d-flex align-items-center group"> Performance Considerations <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="performance-considerations" aria-haspopup="dialog" aria-label="Share link: Performance Considerations"> Share link </button> </h3> <h4 id="wal-write-throughput" class="position-relative d-flex align-items-center group"> WAL Write Throughput <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="wal-write-throughput" aria-haspopup="dialog" aria-label="Share link: WAL Write Throughput"> Share link </button> </h4>Optimize WAL write performance: <ol> <li> Batch commits: Group multiple transactions into single fsync <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">wal: group_commit_delay: 10ms # Wait up to 10ms to batch commits group_commit_size: 100 # Or batch up to 100 transactions </code></pre></div></li> <li> Asynchronous commit: Trade durability for throughput (use cautiously) <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-gql" data-lang="gql">BEGIN TRANSACTION ISOLATION LEVEL READ COMMITTED, COMMIT MODE ASYNC; -- Transaction commits without waiting for fsync COMMIT; </code></pre></div></li> <li> WAL compression: Reduce WAL size with compression <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">wal: compression: lz4 # Fast compression algorithm </code></pre></div></li> </ol> <h4 id="recovery-performance" class="position-relative d-flex align-items-center group"> Recovery Performance <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="recovery-performance" aria-haspopup="dialog" aria-label="Share link: Recovery Performance"> Share link </button> </h4>Optimize recovery speed: <ol> <li>Parallel replay: Replay independent transactions concurrently</li> <li>Incremental checkpoints: Spread checkpoint I/O over time</li> <li>Hot standby: Keep replica always ready for instant failover</li> </ol> <h3 id="related-topics" class="position-relative d-flex align-items-center group"> Related Topics <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="related-topics" aria-haspopup="dialog" aria-label="Share link: Related Topics"> Share link </button> </h3><ul> <li><a href="/tags/acid/" >ACID Transactions</a> - WAL enables durability</li> <li><a href="/tags/mvcc/" >Multi-Version Concurrency Control (MVCC)</a> - WAL integrates with MVCC</li> <li><a href="/docs/recovery/" >Crash Recovery</a> - Recovery process details</li> <li><a href="/tags/distributed/" >Replication</a> - WAL-based replication</li> <li><a href="/docs/operations/backup/" >Backup and Restore</a> - Using WAL for backups</li> <li><a href="/tags/performance/" >Performance</a> - WAL performance tuning</li> </ul> <h3 id="further-reading" class="position-relative d-flex align-items-center group"> Further Reading <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="further-reading" aria-haspopup="dialog" aria-label="Share link: Further Reading"> Share link </button> </h3><ul> <li><a href="/docs/recovery/" >Durability and Recovery Guide</a> - Complete recovery documentation</li> <li><a href="/docs/architecture/" >Architecture Overview</a> - WAL in system architecture</li> <li><a href="/docs/query/performance-tuning/" >Performance Tuning</a> - WAL optimization</li> <li><a href="/docs/operations/monitoring/" >Monitoring Guide</a> - WAL health monitoring</li> </ul> Geode’s Write-Ahead Logging implementation provides rock-solid durability guarantees while maintaining high write throughput—essential for production graph workloads where data loss is unacceptable. <h3 id="advanced-wal-features" class="position-relative d-flex align-items-center group"> Advanced WAL Features <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="advanced-wal-features" aria-haspopup="dialog" aria-label="Share link: Advanced WAL Features"> Share link </button> </h3> <h4 id="parallel-wal-writing" class="position-relative d-flex align-items-center group"> Parallel WAL Writing <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="parallel-wal-writing" aria-haspopup="dialog" aria-label="Share link: Parallel WAL Writing"> Share link </button> </h4>Geode uses parallel WAL writers for improved throughput: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback">Transaction Commit Pipeline: Thread 1: Txn A → WAL Buffer → Flush Queue Thread 2: Txn B → WAL Buffer → Flush Queue Thread 3: Txn C → WAL Buffer → Flush Queue ↓ WAL Writer Thread ↓ fsync() call (batches A, B, C) ↓ All 3 txns commit Benefit: Single fsync() commits multiple transactions Result: 10,000+ commits/sec on NVMe SSD </code></pre></div> <h4 id="group-commit-optimization" class="position-relative d-flex align-items-center group"> Group Commit Optimization <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="group-commit-optimization" aria-haspopup="dialog" aria-label="Share link: Group Commit Optimization"> Share link </button> </h4>Batch commits to amortize fsync() cost: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"># geode.yaml wal: group_commit: enabled: true max_delay_ms: 10 # Wait up to 10ms to batch max_batch_size: 100 # Or commit when 100 txns queued # Result: # - Individual commit latency: 0.1ms (WAL append) # - Actual fsync() latency: 10ms (once per batch) # - Effective throughput: 10,000 commits/sec vs 100 commits/sec </code></pre></div> <h4 id="wal-compression" class="position-relative d-flex align-items-center group"> WAL Compression <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="wal-compression" aria-haspopup="dialog" aria-label="Share link: WAL Compression"> Share link </button> </h4>Reduce WAL storage and I/O: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">wal: compression: algorithm: lz4 # Fast compression level: 1 # Low CPU overhead min_size: 1024 # Only compress records >1KB # Results: # - 60-80% compression ratio for typical graph operations # - <5% CPU overhead for compression # - Reduced network bandwidth for replication </code></pre></div> <h4 id="incremental-checkpointing" class="position-relative d-flex align-items-center group"> Incremental Checkpointing <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="incremental-checkpointing" aria-haspopup="dialog" aria-label="Share link: Incremental Checkpointing"> Share link </button> </h4>Spread checkpoint I/O over time: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback">Traditional Checkpoint: Time: ────────────[CHECKPOINT]──────────────────── I/O: ████████████ (burst) Incremental Checkpoint: Time: ──────────────────────────────────────────── I/O: ██ ██ ██ ██ ██ ██ (spread out) Benefits: - No I/O spikes - Consistent query performance - Faster recovery (shorter WAL replay) </code></pre></div>Configuration: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">wal: checkpoint: mode: incremental target_duration: 300s # Spread over 5 minutes max_dirty_pages: 100000 # Trigger at 100k dirty pages completion_target: 0.9 # Aim to complete 90% of checkpoint interval </code></pre></div> <h3 id="wal-replication-strategies" class="position-relative d-flex align-items-center group"> WAL Replication Strategies <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="wal-replication-strategies" aria-haspopup="dialog" aria-label="Share link: WAL Replication Strategies"> Share link </button> </h3> <h4 id="streaming-replication" class="position-relative d-flex align-items-center group"> Streaming Replication <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="streaming-replication" aria-haspopup="dialog" aria-label="Share link: Streaming Replication"> Share link </button> </h4>Ship WAL records to replicas in real-time: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback">Primary Server Replica Server │ │ │ [Txn Commit] │ │ ↓ │ │ [WAL Write] │ │ ↓ │ │ [Stream WAL Record]───────────────> │ │ [Apply WAL] │ ↓ │ <─────────────[ACK]────────────── [Update State] │ │ │ [Commit Txn] │ </code></pre></div>Implementation: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">replication: mode: streaming sync_mode: async # or 'sync' for synchronous replication max_lag_bytes: 10485760 # 10MB max lag max_lag_seconds: 10 # 10 seconds max time lag replicas: - host: replica1.example.com port: 7000 sync_priority: 1 - host: replica2.example.com port: 7000 sync_priority: 2 </code></pre></div> <h4 id="synchronous-vs-asynchronous-replication" class="position-relative d-flex align-items-center group"> Synchronous vs. Asynchronous Replication <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="synchronous-vs-asynchronous-replication" aria-haspopup="dialog" aria-label="Share link: Synchronous vs. Asynchronous Replication"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-gql" data-lang="gql">-- Synchronous replication (wait for replica ACK) BEGIN TRANSACTION REPLICATION sync; CREATE (:User {id: 123, name: 'Alice'}); COMMIT; -- Blocks until replica confirms WAL applied -- Asynchronous replication (don't wait) BEGIN TRANSACTION REPLICATION async; CREATE (:User {id: 456, name: 'Bob'}); COMMIT; -- Returns immediately, replica applies eventually </code></pre></div>Trade-offs: Synchronous: <ul> <li>Pro: Zero data loss on failover</li> <li>Pro: Strong consistency across replicas</li> <li>Con: Higher commit latency (network round-trip)</li> <li>Con: Availability depends on replica health</li> </ul> Asynchronous: <ul> <li>Pro: Low commit latency</li> <li>Pro: Primary unaffected by replica failures</li> <li>Con: Potential data loss on failover (uncommitted WAL)</li> <li>Con: Eventual consistency</li> </ul> <h4 id="cascading-replication" class="position-relative d-flex align-items-center group"> Cascading Replication <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="cascading-replication" aria-haspopup="dialog" aria-label="Share link: Cascading Replication"> Share link </button> </h4>Replicate across multiple tiers: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-gdscript3" data-lang="gdscript3"> Primary (US-East) │ ├───> Replica 1 (US-East) │ └───> Replica 2 (US-West) │ ├───> Replica 3 (EU-West) │ └───> Replica 4 (AP-South) Benefits: - Reduced load on primary - Geographic distribution - Lower cross-region bandwidth </code></pre></div> <h3 id="wal-based-logical-replication" class="position-relative d-flex align-items-center group"> WAL-Based Logical Replication <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="wal-based-logical-replication" aria-haspopup="dialog" aria-label="Share link: WAL-Based Logical Replication"> Share link </button> </h3>Replicate specific subsets of data: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-gql" data-lang="gql">-- Create logical replication slot CREATE REPLICATION SLOT analytics_slot LOGICAL; -- Define publication (what to replicate) CREATE PUBLICATION analytics_pub FOR LABELS (User, Order, Product) WHERE region = 'us-east'; -- Subscriber consumes filtered WAL stream SUBSCRIBE TO PUBLICATION analytics_pub FROM SLOT analytics_slot; </code></pre></div>Use cases: <ul> <li>Replicate subset of data to analytics database</li> <li>Multi-tenant replication (one tenant per replica)</li> <li>Cross-database replication (Geode → PostgreSQL)</li> </ul> <h3 id="disaster-recovery-with-wal" class="position-relative d-flex align-items-center group"> Disaster Recovery with WAL <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="disaster-recovery-with-wal" aria-haspopup="dialog" aria-label="Share link: Disaster Recovery with WAL"> Share link </button> </h3> <h4 id="continuous-wal-archival" class="position-relative d-flex align-items-center group"> Continuous WAL Archival <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="continuous-wal-archival" aria-haspopup="dialog" aria-label="Share link: Continuous WAL Archival"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash">#!/bin/bash # WAL archival script # Geode calls this when WAL segment complete WAL_FILE=$1 WAL_PATH="/var/lib/geode/wal/$WAL_FILE" # Archive to S3 aws s3 cp "$WAL_PATH" "s3://geode-wal-archive/$(date +%Y%m%d)/$WAL_FILE" \ --storage-class GLACIER # Verify upload if [ $? -eq 0 ]; then echo "Archived $WAL_FILE to S3" else echo "Failed to archive $WAL_FILE" >&2 exit 1 fi </code></pre></div>Configuration: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">wal: archive: enabled: true command: "/usr/local/bin/archive_wal.sh %p" timeout: 300 # 5 minute timeout retention_days: 30 </code></pre></div> <h4 id="point-in-time-recovery-pitr" class="position-relative d-flex align-items-center group"> Point-in-Time Recovery (PITR) <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="point-in-time-recovery-pitr" aria-haspopup="dialog" aria-label="Share link: Point-in-Time Recovery (PITR)"> Share link </button> </h4>Restore database to specific timestamp: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"># Stop Geode server systemctl stop geode # Restore base backup tar -xzf /backups/base-backup-20260124.tar.gz -C /var/lib/geode/data # Configure recovery cat > /var/lib/geode/data/recovery.conf <<EOF restore_command = 'aws s3 cp s3://geode-wal-archive/%f %p' recovery_target_time = '2026-01-24 14:30:00' recovery_target_action = promote EOF # Start Geode (recovery mode) systemctl start geode # Geode replays WAL up to target time, then becomes operational </code></pre></div>Recovery process: <ol> <li>Restore base backup</li> <li>Replay WAL files from archive</li> <li>Stop at recovery target time</li> <li>Apply final checkpoint</li> <li>Database ready at target timestamp</li> </ol> <h4 id="wal-retention-for-compliance" class="position-relative d-flex align-items-center group"> WAL Retention for Compliance <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="wal-retention-for-compliance" aria-haspopup="dialog" aria-label="Share link: WAL Retention for Compliance"> Share link </button> </h4>Configure long-term WAL retention: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">wal: archive: enabled: true retention: short_term: days: 7 storage: local # /var/lib/geode/wal_archive long_term: days: 2555 # 7 years for compliance storage: s3_glacier bucket: geode-compliance-archive </code></pre></div> <h3 id="monitoring-wal-health-1" class="position-relative d-flex align-items-center group"> Monitoring WAL Health <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="monitoring-wal-health-1" aria-haspopup="dialog" aria-label="Share link: Monitoring WAL Health"> Share link </button> </h3> <h4 id="critical-wal-metrics" class="position-relative d-flex align-items-center group"> Critical WAL Metrics <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="critical-wal-metrics" aria-haspopup="dialog" aria-label="Share link: Critical WAL Metrics"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python">from prometheus_client import Gauge, Counter # WAL metrics wal_current_lsn = Gauge('geode_wal_current_lsn', 'Current WAL LSN') wal_files_count = Gauge('geode_wal_files_count', 'Number of WAL files') wal_bytes_written = Counter('geode_wal_bytes_written_total', 'Total bytes written to WAL') wal_sync_duration = Histogram('geode_wal_sync_seconds', 'WAL fsync duration') # Replication lag replication_lag_bytes = Gauge( 'geode_replication_lag_bytes', 'Replication lag in bytes', ['replica'] ) replication_lag_seconds = Gauge( 'geode_replication_lag_seconds', 'Replication lag in seconds', ['replica'] ) # Checkpoint metrics checkpoint_duration = Histogram('geode_checkpoint_duration_seconds', 'Checkpoint duration') checkpoint_pages_written = Counter('geode_checkpoint_pages_written_total', 'Pages written during checkpoint') </code></pre></div> <h4 id="wal-health-checks" class="position-relative d-flex align-items-center group"> WAL Health Checks <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="wal-health-checks" aria-haspopup="dialog" aria-label="Share link: WAL Health Checks"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python">async def check_wal_health(client): """Monitor WAL health""" # Check WAL disk usage wal_size, _ = await client.query(""" SELECT SUM(size_bytes) AS total_size FROM SYSTEM.wal_files """) if wal_size > 10 * 1024 * 1024 * 1024: # >10GB alert("WAL directory growing too large") # Check replication lag lag, _ = await client.query(""" SELECT replica_name, current_lsn - replica_lsn AS lag_bytes, extract(epoch from now() - last_replay_time) AS lag_seconds FROM SYSTEM.replication_status """) for replica in lag.bindings: if replica['lag_bytes'] > 100 * 1024 * 1024: # >100MB alert(f"Replica {replica['replica_name']} falling behind") # Check WAL archival unarchived, _ = await client.query(""" SELECT COUNT(*) AS count FROM SYSTEM.wal_files WHERE archived = false AND created_at < NOW() - INTERVAL '1 hour' """) if unarchived.bindings[0]['count'] > 10: alert("WAL archival falling behind") </code></pre></div> <h4 id="grafana-dashboard-queries" class="position-relative d-flex align-items-center group"> Grafana Dashboard Queries <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="grafana-dashboard-queries" aria-haspopup="dialog" aria-label="Share link: Grafana Dashboard Queries"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-promql" data-lang="promql"># WAL write rate rate(geode_wal_bytes_written_total[5m]) # Average fsync duration rate(geode_wal_sync_seconds_sum[5m]) / rate(geode_wal_sync_seconds_count[5m]) # Replication lag (seconds) geode_replication_lag_seconds{replica="replica1"} # WAL disk usage geode_wal_files_count * 64 * 1024 * 1024 # Assuming 64MB segments </code></pre></div> <h3 id="further-reading-1" class="position-relative d-flex align-items-center group"> Further Reading <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="further-reading-1" aria-haspopup="dialog" aria-label="Share link: Further Reading"> Share link </button> </h3><ul> <li><a href="/docs/recovery/" >Durability and Recovery Guide</a> - Complete recovery documentation</li> <li><a href="/docs/architecture/" >Architecture Overview</a> - WAL in system architecture</li> <li><a href="/docs/query/performance-tuning/" >Performance Tuning</a> - WAL optimization</li> <li><a href="/docs/operations/monitoring/" >Monitoring Guide</a> - WAL health monitoring</li> <li><a href="https://www.postgresql.org/docs/current/wal-internals.html" aria-label="PostgreSQL WAL Internals – opens in new window" target="_blank" rel="noopener noreferrer" >PostgreSQL WAL Internals ↗ </a> - Detailed WAL concepts</li> <li><a href="https://cs.stanford.edu/people/chrismre/cs345/rl/aries.pdf" aria-label="ARIES Recovery Algorithm – opens in new window" target="_blank" rel="noopener noreferrer" >ARIES Recovery Algorithm ↗ </a> - Foundational paper</li> </ul> Geode’s Write-Ahead Logging implementation provides rock-solid durability guarantees while maintaining high write throughput—essential for production graph workloads where data loss is unacceptable.

Popular

Related Articles

Transactions and Data Integrity