Monitoring and Telemetry | Geode Database

<h2 id="monitoring-and-telemetry" class="position-relative d-flex align-items-center group"> Monitoring and Telemetry <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="monitoring-and-telemetry" aria-haspopup="dialog" aria-label="Share link: Monitoring and Telemetry"> Share link </button> </h2><div id="headingShareModal" class="heading-share-modal" role="dialog" aria-modal="true" aria-labelledby="headingShareTitle" hidden> <div class="hsm-dialog" role="document"> <div class="hsm-header"> <h2 id="headingShareTitle" class="h6 mb-0 fw-bold">Share this section</h2> <button type="button" class="hsm-close" aria-label="Close"> </button> </div> <div class="hsm-body"> <label for="headingShareInput" class="form-label small text-muted mb-1 text-uppercase fw-bold" style="font-size: 0.7rem; letter-spacing: 0.5px;">Permalink</label> <div class="input-group mb-4 hsm-url-group"> <input id="headingShareInput" type="text" class="form-control font-monospace" readonly aria-readonly="true" style="font-size: 0.85rem;" /> <button class="btn btn-primary hsm-copy" type="button" aria-label="Copy" title="Copy"> </button> </div> <div class="small fw-bold mb-2 text-muted text-uppercase" style="font-size: 0.7rem; letter-spacing: 0.5px;">Share via</div> <div class="hsm-share-grid"> <a id="share-twitter" class="btn btn-outline-secondary w-100" target="_blank" rel="noopener noreferrer"> Twitter </a> <a id="share-linkedin" class="btn btn-outline-secondary w-100" target="_blank" rel="noopener noreferrer"> LinkedIn </a> <a id="share-facebook" class="btn btn-outline-secondary w-100" target="_blank" rel="noopener noreferrer"> Facebook </a> </div> </div> </div> </div> <style> .heading-share-modal { position: fixed; inset: 0; display: flex; justify-content: center; align-items: center; background: rgba(0, 0, 0, 0.6); z-index: 1050; padding: 1rem; backdrop-filter: blur(4px); -webkit-backdrop-filter: blur(4px); } .heading-share-modal[hidden] { display: none !important; } .hsm-dialog { max-width: 420px; width: 100%; background: var(--bs-body-bg, #fff); color: var(--bs-body-color, #212529); border: 1px solid var(--bs-border-color, rgba(0,0,0,0.1)); border-radius: 1rem; box-shadow: 0 25px 50px -12px rgba(0, 0, 0, 0.25); overflow: hidden; animation: hsm-fade-in 0.2s ease-out; } @keyframes hsm-fade-in { from { opacity: 0; transform: scale(0.95); } to { opacity: 1; transform: scale(1); } } [data-bs-theme="dark"] .hsm-dialog { background: #1e293b; border-color: rgba(255,255,255,0.1); color: #f8f9fa; } .hsm-header { display: flex; justify-content: space-between; align-items: center; padding: 1rem 1.5rem; border-bottom: 1px solid var(--bs-border-color, rgba(0,0,0,0.1)); background: rgba(0,0,0,0.02); } [data-bs-theme="dark"] .hsm-header { background: rgba(255,255,255,0.02); border-color: rgba(255,255,255,0.1); } .hsm-close { background: transparent; border: none; color: inherit; opacity: 0.5; padding: 0.25rem 0.5rem; border-radius: 0.25rem; font-size: 1.2rem; line-height: 1; transition: opacity 0.2s; } .hsm-close:hover { opacity: 1; } .hsm-body { padding: 1.5rem; } .hsm-url-group { display: flex !important; align-items: stretch; } .hsm-url-group .form-control { flex: 1; min-width: 0; margin: 0; background: var(--bs-secondary-bg, #f8f9fa); border-color: var(--bs-border-color, #dee2e6); border-top-right-radius: 0; border-bottom-right-radius: 0; height: 42px; } .hsm-url-group .btn { flex: 0 0 auto; margin: 0; margin-left: -1px; border-top-left-radius: 0; border-bottom-left-radius: 0; height: 42px; display: flex; align-items: center; justify-content: center; padding: 0 1.25rem; z-index: 2; } [data-bs-theme="dark"] .hsm-url-group .form-control { background: #0f172a; border-color: #334155; color: #e2e8f0; } .hsm-share-grid { display: flex; flex-direction: column; gap: 0.5rem; } .hsm-share-grid .btn { display: flex; align-items: center; justify-content: center; font-size: 0.9rem; padding: 0.6rem; border-color: var(--bs-border-color); width: 100%; } [data-bs-theme="dark"] .hsm-share-grid .btn { color: #e2e8f0; border-color: #475569; } [data-bs-theme="dark"] .hsm-share-grid .btn:hover { background: #334155; border-color: #cbd5e1; } </style> <script> (function(){ const modal = document.getElementById('headingShareModal'); if(!modal) return; const input = modal.querySelector('#headingShareInput'); const copyBtn = modal.querySelector('.hsm-copy'); const twitter = modal.querySelector('#share-twitter'); const linkedin = modal.querySelector('#share-linkedin'); const facebook = modal.querySelector('#share-facebook'); const closeBtn = modal.querySelector('.hsm-close'); let lastFocus=null; let trapBound=false; function buildUrl(id){ return window.location.origin + window.location.pathname + '#' + id; } function isOpen(){ return !modal.hasAttribute('hidden'); } function hydrate(id){ const url=buildUrl(id); input.value=url; const enc=encodeURIComponent(url); const text=encodeURIComponent(document.title); if(twitter) twitter.href=`https://twitter.com/intent/tweet?url=${enc}&text=${text}`; if(linkedin) linkedin.href=`https://www.linkedin.com/sharing/share-offsite/?url=${enc}`; if(facebook) facebook.href=`https://www.facebook.com/sharer/sharer.php?u=${enc}`; } function openModal(id){ lastFocus=document.activeElement; hydrate(id); if(!isOpen()){ modal.removeAttribute('hidden'); } requestAnimationFrame(()=>{ input.focus(); }); trapFocus(); } function closeModal(){ if(!isOpen()) return; modal.setAttribute('hidden',''); if(lastFocus && typeof lastFocus.focus==='function') lastFocus.focus(); } function copyCurrent(){ try{ navigator.clipboard.writeText(input.value).then(()=>feedback(true),()=>fallback()); } catch(e){ fallback(); } } function fallback(){ input.select(); try{ document.execCommand('copy'); feedback(true);}catch(e){ feedback(false);} } function feedback(ok){ if(!copyBtn) return; const icon=copyBtn.querySelector('i'); if(!icon) return; const prev=copyBtn.getAttribute('data-prev')||icon.className; if(!copyBtn.getAttribute('data-prev')) copyBtn.setAttribute('data-prev',prev); icon.className= ok ? 'fa-duotone fa-clipboard-check':'fa-duotone fa-circle-exclamation'; setTimeout(()=>{ icon.className=prev; },1800); } function handleShareClick(e){ e.preventDefault(); const btn=e.currentTarget; const id=btn.getAttribute('data-share-target'); if(id) openModal(id); } function bindShareButtons(){ document.querySelectorAll('.h-share').forEach(btn=>{ if(!btn.dataset.hShareBound){ btn.addEventListener('click', handleShareClick); btn.dataset.hShareBound='1'; } }); } bindShareButtons(); if(document.readyState==='loading'){ document.addEventListener('DOMContentLoaded', bindShareButtons); } else { requestAnimationFrame(bindShareButtons); } document.addEventListener('click', function(e){ const shareBtn=e.target.closest && e.target.closest('.h-share'); if(shareBtn && !shareBtn.dataset.hShareBound){ handleShareClick.call(shareBtn, e); } }, true); document.addEventListener('click', e=>{ if(e.target===modal) closeModal(); if(e.target.closest && e.target.closest('.hsm-close')){ e.preventDefault(); closeModal(); } if(copyBtn && (e.target===copyBtn || (e.target.closest && e.target.closest('.hsm-copy')))) { e.preventDefault(); copyCurrent(); } }); document.addEventListener('keydown', e=>{ if(e.key==='Escape' && isOpen()) closeModal(); }); function trapFocus(){ if(trapBound) return; trapBound=true; modal.addEventListener('keydown', f=>{ if(f.key==='Tab' && isOpen()){ const focusable=[...modal.querySelectorAll('a[href],button,input,textarea,select,[tabindex]:not([tabindex="-1"])')].filter(el=>!el.hasAttribute('disabled')); if(!focusable.length) return; const first=focusable[0]; const last=focusable[focusable.length-1]; if(f.shiftKey && document.activeElement===first){ f.preventDefault(); last.focus(); } else if(!f.shiftKey && document.activeElement===last){ f.preventDefault(); first.focus(); } } }); } if(closeBtn) closeBtn.addEventListener('click', e=>{ e.preventDefault(); closeModal(); }); })(); </script>Comprehensive observability for Geode: health checks, Prometheus metrics, optional telemetry, and audit logging. <h3 id="health-and-readiness-endpoints" class="position-relative d-flex align-items-center group"> Health and Readiness Endpoints <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="health-and-readiness-endpoints" aria-haspopup="dialog" aria-label="Share link: Health and Readiness Endpoints"> Share link </button> </h3>From <code>USAGE.md</code>: <h4 id="health-check" class="position-relative d-flex align-items-center group"> Health Check <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="health-check" aria-haspopup="dialog" aria-label="Share link: Health Check"> Share link </button> </h4>Endpoint: <code>GET /health</code> Purpose: Verify server is running and responsive <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash">curl -f http://localhost:8080/health || echo "Health check failed" </code></pre></div>Response (healthy): <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-json" data-lang="json">{ "status": "healthy", "timestamp": "2024-01-15T14:30:00Z", "version": "0.2.18" } </code></pre></div>Response (unhealthy): <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-json" data-lang="json">{ "status": "unhealthy", "error": "database connection failed", "timestamp": "2024-01-15T14:30:00Z" } </code></pre></div>HTTP status codes: <ul> <li><code>200 OK</code> - Healthy</li> <li><code>503 Service Unavailable</code> - Unhealthy</li> </ul> <h4 id="readiness-check" class="position-relative d-flex align-items-center group"> Readiness Check <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="readiness-check" aria-haspopup="dialog" aria-label="Share link: Readiness Check"> Share link </button> </h4>Endpoint: <code>GET /ready</code> Purpose: Verify server is ready to accept traffic (useful for Kubernetes liveness probes) <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash">curl -f http://localhost:8080/ready </code></pre></div>Response (ready): <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-json" data-lang="json">{ "status": "ready", "timestamp": "2024-01-15T14:30:00Z", "connections": 42, "active_transactions": 3 } </code></pre></div>Response (not ready): <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-json" data-lang="json">{ "status": "not_ready", "reason": "initialization in progress", "timestamp": "2024-01-15T14:30:00Z" } </code></pre></div>Kubernetes liveness probe: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">livenessProbe: httpGet: path: /health port: 8080 initialDelaySeconds: 30 periodSeconds: 10 </code></pre></div>Kubernetes readiness probe: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">readinessProbe: httpGet: path: /ready port: 8080 initialDelaySeconds: 5 periodSeconds: 5 </code></pre></div> <h3 id="prometheus-metrics" class="position-relative d-flex align-items-center group"> Prometheus Metrics <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="prometheus-metrics" aria-haspopup="dialog" aria-label="Share link: Prometheus Metrics"> Share link </button> </h3>From <code>USAGE.md</code>: <h4 id="metrics-endpoint" class="position-relative d-flex align-items-center group"> Metrics Endpoint <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="metrics-endpoint" aria-haspopup="dialog" aria-label="Share link: Metrics Endpoint"> Share link </button> </h4>Endpoint: <code>GET /metrics</code> Format: Prometheus text exposition format <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash">curl http://localhost:8080/metrics </code></pre></div> <h4 id="key-metrics" class="position-relative d-flex align-items-center group"> Key Metrics <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="key-metrics" aria-haspopup="dialog" aria-label="Share link: Key Metrics"> Share link </button> </h4>From <code>USAGE.md</code>, key metrics list: <h5 id="query-metrics" class="position-relative d-flex align-items-center group"> Query Metrics <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="query-metrics" aria-haspopup="dialog" aria-label="Share link: Query Metrics"> Share link </button> </h5><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-gdscript3" data-lang="gdscript3"># HELP geode_queries_total Total number of queries executed # TYPE geode_queries_total counter geode_queries_total{graph="SocialNetwork",status="success"} 12345 # HELP geode_query_duration_seconds Query execution time # TYPE geode_query_duration_seconds histogram geode_query_duration_seconds_bucket{graph="SocialNetwork",le="0.001"} 100 geode_query_duration_seconds_bucket{graph="SocialNetwork",le="0.01"} 500 geode_query_duration_seconds_bucket{graph="SocialNetwork",le="0.1"} 1200 geode_query_duration_seconds_bucket{graph="SocialNetwork",le="1.0"} 1250 geode_query_duration_seconds_sum{graph="SocialNetwork"} 45.67 geode_query_duration_seconds_count{graph="SocialNetwork"} 1250 # HELP geode_query_errors_total Total number of query errors # TYPE geode_query_errors_total counter geode_query_errors_total{graph="SocialNetwork",error_type="syntax"} 12 geode_query_errors_total{graph="SocialNetwork",error_type="constraint_violation"} 5 </code></pre></div> <h5 id="transaction-metrics" class="position-relative d-flex align-items-center group"> Transaction Metrics <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="transaction-metrics" aria-haspopup="dialog" aria-label="Share link: Transaction Metrics"> Share link </button> </h5><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-gdscript3" data-lang="gdscript3"># HELP geode_transactions_active Currently active transactions # TYPE geode_transactions_active gauge geode_transactions_active 3 # HELP geode_transactions_committed Total committed transactions # TYPE geode_transactions_committed counter geode_transactions_committed 5678 # HELP geode_transactions_aborted Total aborted transactions # TYPE geode_transactions_aborted counter geode_transactions_aborted{reason="serialization_error"} 23 geode_transactions_aborted{reason="constraint_violation"} 15 # HELP geode_transaction_duration_seconds Transaction execution time # TYPE geode_transaction_duration_seconds histogram geode_transaction_duration_seconds_bucket{le="0.1"} 4500 geode_transaction_duration_seconds_bucket{le="1.0"} 5600 geode_transaction_duration_seconds_bucket{le="10.0"} 5670 </code></pre></div> <h5 id="storage-metrics" class="position-relative d-flex align-items-center group"> Storage Metrics <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="storage-metrics" aria-haspopup="dialog" aria-label="Share link: Storage Metrics"> Share link </button> </h5><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback"># HELP geode_storage_pages_total Total number of data pages # TYPE geode_storage_pages_total gauge geode_storage_pages_total 1234567 # HELP geode_storage_cache_hits Cache hit count # TYPE geode_storage_cache_hits counter geode_storage_cache_hits 98765432 # HELP geode_storage_cache_misses Cache miss count # TYPE geode_storage_cache_misses counter geode_storage_cache_misses 1234567 # HELP geode_storage_cache_hit_ratio Cache hit ratio # TYPE geode_storage_cache_hit_ratio gauge geode_storage_cache_hit_ratio 0.987 # HELP geode_wal_writes_bytes Write-Ahead Log bytes written # TYPE geode_wal_writes_bytes counter geode_wal_writes_bytes 123456789012 </code></pre></div> <h5 id="connection-metrics" class="position-relative d-flex align-items-center group"> Connection Metrics <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="connection-metrics" aria-haspopup="dialog" aria-label="Share link: Connection Metrics"> Share link </button> </h5><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback"># HELP geode_connections_active Currently active client connections # TYPE geode_connections_active gauge geode_connections_active 42 # HELP geode_connections_total Total connections since start # TYPE geode_connections_total counter geode_connections_total 12345 </code></pre></div> <h5 id="index-metrics" class="position-relative d-flex align-items-center group"> Index Metrics <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="index-metrics" aria-haspopup="dialog" aria-label="Share link: Index Metrics"> Share link </button> </h5><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback"># HELP geode_index_lookups_total Index lookup count # TYPE geode_index_lookups_total counter geode_index_lookups_total{index="person_age_idx",type="btree"} 45678 # HELP geode_index_size_bytes Index size in bytes # TYPE geode_index_size_bytes gauge geode_index_size_bytes{index="person_age_idx",type="btree"} 12345678 </code></pre></div> <h4 id="prometheus-configuration" class="position-relative d-flex align-items-center group"> Prometheus Configuration <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="prometheus-configuration" aria-haspopup="dialog" aria-label="Share link: Prometheus Configuration"> Share link </button> </h4>prometheus.yml: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">global: scrape_interval: 15s evaluation_interval: 15s scrape_configs: - job_name: 'geode' static_configs: - targets: ['localhost:8080'] metrics_path: '/metrics' </code></pre></div>Start Prometheus: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash">prometheus --config.file=prometheus.yml </code></pre></div>Query examples: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-promql" data-lang="promql"># Query rate (QPS) rate(geode_queries_total[5m]) # 95th percentile query latency histogram_quantile(0.95, rate(geode_query_duration_seconds_bucket[5m])) # Cache hit ratio geode_storage_cache_hit_ratio # Transaction abort rate rate(geode_transactions_aborted[5m]) </code></pre></div> <h3 id="optional-telemetry" class="position-relative d-flex align-items-center group"> Optional Telemetry <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="optional-telemetry" aria-haspopup="dialog" aria-label="Share link: Optional Telemetry"> Share link </button> </h3>From <code>TELEMETRY.md</code>: Paging telemetry outputs detailed system events to stderr (JSONL format). Purpose: Debugging, CI/testing, performance analysis Enable: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"># Enable paging telemetry export GEODE_TELEMETRY_PAGING=1 ./geode serve </code></pre></div>Output (stderr, JSONL): <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-json" data-lang="json">{"event":"page_read","page_id":12345,"timestamp":"2024-01-15T14:30:00.123Z","duration_us":42} {"event":"page_write","page_id":12346,"timestamp":"2024-01-15T14:30:00.456Z","duration_us":156} {"event":"index_lookup","index":"person_age_idx","key":30,"timestamp":"2024-01-15T14:30:00.789Z","duration_us":23} </code></pre></div>CI/Testing toggles (from <code>TELEMETRY.md</code>): <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"># Disable telemetry in tests (reduce noise) export GEODE_TELEMETRY_PAGING=0 # Enable verbose telemetry for debugging export GEODE_TELEMETRY_VERBOSE=1 </code></pre></div>Note: Paging telemetry has performance overhead (~5-10%). Use only for debugging/testing. <h3 id="audit-logs-and-tracing" class="position-relative d-flex align-items-center group"> Audit Logs and Tracing <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="audit-logs-and-tracing" aria-haspopup="dialog" aria-label="Share link: Audit Logs and Tracing"> Share link </button> </h3>From <code>AUDIT_LOGGING.md</code>: <h4 id="audit-log-configuration" class="position-relative d-flex align-items-center group"> Audit Log Configuration <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="audit-log-configuration" aria-haspopup="dialog" aria-label="Share link: Audit Log Configuration"> Share link </button> </h4>Enable audit logging (<code>geode.yaml</code>): <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">security: audit: enabled: true log_path: "/var/log/geode/audit.jsonl" # Syslog/CEF forwarding syslog: enabled: true address: "syslog.example.com:514" format: "CEF" # Common Event Format # Retention retention_days: 365 max_size: "10GB" </code></pre></div> <h4 id="whats-logged" class="position-relative d-flex align-items-center group"> What&rsquo;s Logged <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="whats-logged" aria-haspopup="dialog" aria-label="Share link: Whats Logged"> Share link </button> </h4>Events logged: <ul> <li>✅ Authentication (login, logout, failed attempts)</li> <li>✅ Authorization decisions (policy evaluations)</li> <li>✅ Schema changes (CREATE/ALTER/DROP)</li> <li>✅ Administrative actions (user/role management)</li> <li>✅ Query metadata (timestamp, user, graph, execution time)</li> </ul> Events NOT logged (privacy/security): <ul> <li>❌ Query text (avoid logging sensitive data)</li> <li>❌ Query parameters</li> <li>❌ Result sets</li> </ul> <h4 id="audit-log-entry-format" class="position-relative d-flex align-items-center group"> Audit Log Entry Format <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="audit-log-entry-format" aria-haspopup="dialog" aria-label="Share link: Audit Log Entry Format"> Share link </button> </h4>Example (JSONL): <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-json" data-lang="json">{ "timestamp": "2024-01-15T14:30:00.123Z", "event_type": "query_executed", "user": "alice", "graph": "SocialNetwork", "session_id": "550e8400-e29b-41d4-a716-446655440000", "trace_id": "7c9e8d6f-5b4a-3c2d-1e0f-9a8b7c6d5e4f", "execution_time_ms": 23.5, "rows_returned": 150, "prev_log_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "signature": "3045022100..." } </code></pre></div>Tamper-evident properties: <ul> <li>Hash chain: Each entry includes <code>prev_log_hash</code> (SHA-256 of previous entry)</li> <li>Signatures: Entries signed with server private key for non-repudiation</li> <li>Tracing IDs: <code>trace_id</code> correlates events across distributed systems</li> </ul> <h4 id="tracing-ids" class="position-relative d-flex align-items-center group"> Tracing IDs <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="tracing-ids" aria-haspopup="dialog" aria-label="Share link: Tracing IDs"> Share link </button> </h4>Purpose: Correlate events across: <ul> <li>Multiple queries in a transaction</li> <li>Distributed query execution (federated queries)</li> <li>CDC webhooks and downstream processing</li> </ul> Example: Distributed trace: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-json" data-lang="json">// Client query {"trace_id": "7c9e8d6f-5b4a-3c2d-1e0f-9a8b7c6d5e4f", "event": "query_started", ...} // Federated execution on shard 1 {"trace_id": "7c9e8d6f-5b4a-3c2d-1e0f-9a8b7c6d5e4f", "event": "shard_query", "shard": 1, ...} // Federated execution on shard 2 {"trace_id": "7c9e8d6f-5b4a-3c2d-1e0f-9a8b7c6d5e4f", "event": "shard_query", "shard": 2, ...} // Result merge {"trace_id": "7c9e8d6f-5b4a-3c2d-1e0f-9a8b7c6d5e4f", "event": "result_merged", ...} // Query completed {"trace_id": "7c9e8d6f-5b4a-3c2d-1e0f-9a8b7c6d5e4f", "event": "query_completed", ...} </code></pre></div> <h4 id="audit-log-analysis" class="position-relative d-flex align-items-center group"> Audit Log Analysis <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="audit-log-analysis" aria-haspopup="dialog" aria-label="Share link: Audit Log Analysis"> Share link </button> </h4>Verify hash chain integrity: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"># Python script to verify audit log integrity import json import hashlib prev_hash = None with open('/var/log/geode/audit.jsonl') as f: for line_num, line in enumerate(f, 1): entry = json.loads(line) # Compute expected hash if prev_hash is not None: if entry['prev_log_hash'] != prev_hash: print(f"Hash chain broken at line {line_num}!") exit(1) # Update for next iteration prev_hash = hashlib.sha256(line.encode()).hexdigest() print("Audit log integrity verified") </code></pre></div>Query audit logs: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"># Find all queries by user 'alice' cat /var/log/geode/audit.jsonl | jq 'select(.user == "alice")' # Find failed authentication attempts cat /var/log/geode/audit.jsonl | jq 'select(.event_type == "auth_failed")' # Find slow queries (>1 second) cat /var/log/geode/audit.jsonl | jq 'select(.execution_time_ms > 1000)' </code></pre></div> <h3 id="logging-configuration" class="position-relative d-flex align-items-center group"> Logging Configuration <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="logging-configuration" aria-haspopup="dialog" aria-label="Share link: Logging Configuration"> Share link </button> </h3>From <code>DOCKER_LOG_LEVEL_CONFIG.md</code>: <h4 id="log-levels" class="position-relative d-flex align-items-center group"> Log Levels <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="log-levels" aria-haspopup="dialog" aria-label="Share link: Log Levels"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"># Set log level (debug/info/warn/error) export LOG_LEVEL=info ./geode serve </code></pre></div>Docker (<code>docker-compose.yml</code>): <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">services: geode: image: geodedb/geode:latest environment: LOG_LEVEL: info # or debug, warn, error </code></pre></div>Defaults: <ul> <li>Development: <code>debug</code></li> <li>Production: <code>info</code></li> </ul> <h4 id="log-formats" class="position-relative d-flex align-items-center group"> Log Formats <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="log-formats" aria-haspopup="dialog" aria-label="Share link: Log Formats"> Share link </button> </h4>Text format (human-readable): <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback">2024-01-15T14:30:00.123Z INFO [geode::server] Server started on 0.0.0.0:3141 2024-01-15T14:30:01.456Z DEBUG [geode::query] Executing query for user 'alice' 2024-01-15T14:30:01.789Z WARN [geode::storage] Cache miss for page 12345 </code></pre></div>JSON format (machine-readable): <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-json" data-lang="json">{"timestamp":"2024-01-15T14:30:00.123Z","level":"INFO","module":"geode::server","message":"Server started on 0.0.0.0:3141"} {"timestamp":"2024-01-15T14:30:01.456Z","level":"DEBUG","module":"geode::query","message":"Executing query for user 'alice'"} </code></pre></div>Configure (<code>geode.yaml</code>): <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">logging: level: 'info' format: 'json' # or 'text' </code></pre></div> <h3 id="grafana-dashboards" class="position-relative d-flex align-items-center group"> Grafana Dashboards <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="grafana-dashboards" aria-haspopup="dialog" aria-label="Share link: Grafana Dashboards"> Share link </button> </h3>From <code>deployment/DEPLOYMENT.md</code>: Access Grafana: <code>http://localhost:3000</code> (admin/admin) <h4 id="import-geode-dashboard" class="position-relative d-flex align-items-center group"> Import Geode Dashboard <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="import-geode-dashboard" aria-haspopup="dialog" aria-label="Share link: Import Geode Dashboard"> Share link </button> </h4><ol> <li>Download dashboard JSON from <code>docs/deployment/grafana-dashboard.json</code></li> <li>Import in Grafana: Home → Dashboards → Import</li> <li>Configure data source: Select Prometheus</li> </ol> <h4 id="dashboard-panels" class="position-relative d-flex align-items-center group"> Dashboard Panels <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="dashboard-panels" aria-haspopup="dialog" aria-label="Share link: Dashboard Panels"> Share link </button> </h4>Query Performance: <ul> <li>Query rate (QPS)</li> <li>50th/95th/99th percentile latency</li> <li>Error rate</li> </ul> Transaction Metrics: <ul> <li>Active transactions</li> <li>Commit/abort rate</li> <li>Serialization error rate</li> </ul> Storage: <ul> <li>Cache hit ratio</li> <li>Page read/write rate</li> <li>WAL write throughput</li> </ul> Connections: <ul> <li>Active connections</li> <li>Connection rate</li> </ul> Alerts: <ul> <li>High error rate (>1%)</li> <li>High latency (p95 >1s)</li> <li>Cache hit ratio low (<80%)</li> <li>Transaction abort rate high (>10%)</li> </ul> <h3 id="loki-log-aggregation" class="position-relative d-flex align-items-center group"> Loki Log Aggregation <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="loki-log-aggregation" aria-haspopup="dialog" aria-label="Share link: Loki Log Aggregation"> Share link </button> </h3>From <code>deployment/DEPLOYMENT.md</code>: Loki + Promtail aggregate logs from all Geode instances. Promtail configuration (<code>promtail-config.yaml</code>): <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">server: http_listen_port: 9080 grpc_listen_port: 0 clients: - url: http://loki:3100/loki/api/v1/push scrape_configs: - job_name: geode static_configs: - targets: - localhost labels: job: geode __path__: /var/log/geode/*.log </code></pre></div>Query logs in Grafana: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback"># All logs from geode {job="geode"} # Error logs only {job="geode"} |= "ERROR" # Query execution logs {job="geode"} |= "query_executed" # Slow queries {job="geode"} | json | execution_time_ms > 1000 </code></pre></div> <h3 id="next-steps" class="position-relative d-flex align-items-center group"> Next Steps <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="next-steps" aria-haspopup="dialog" aria-label="Share link: Next Steps"> Share link </button> </h3><ul> <li><a href="/docs/ops/deployment" >Deployment Guide</a> - Production stack setup</li> <li><a href="/docs/security/overview" >Security Guide</a> - Audit logging and integrity</li> <li><a href="/docs/architecture/performance-and-scaling" >Performance and Scaling</a> - Performance metrics interpretation</li> <li><a href="/docs/ops/telemetry-advanced/" >Telemetry Reference</a> - Complete telemetry documentation</li> </ul>