Benchmarking | Geode Database

<h2 id="benchmarking-guide" class="position-relative d-flex align-items-center group"> Benchmarking Guide <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="benchmarking-guide" aria-haspopup="dialog" aria-label="Share link: Benchmarking Guide"> Share link </button> </h2><div id="headingShareModal" class="heading-share-modal" role="dialog" aria-modal="true" aria-labelledby="headingShareTitle" hidden> <div class="hsm-dialog" role="document"> <div class="hsm-header"> <h2 id="headingShareTitle" class="h6 mb-0 fw-bold">Share this section</h2> <button type="button" class="hsm-close" aria-label="Close"> </button> </div> <div class="hsm-body"> <label for="headingShareInput" class="form-label small text-muted mb-1 text-uppercase fw-bold" style="font-size: 0.7rem; letter-spacing: 0.5px;">Permalink</label> <div class="input-group mb-4 hsm-url-group"> <input id="headingShareInput" type="text" class="form-control font-monospace" readonly aria-readonly="true" style="font-size: 0.85rem;" /> <button class="btn btn-primary hsm-copy" type="button" aria-label="Copy" title="Copy"> </button> </div> <div class="small fw-bold mb-2 text-muted text-uppercase" style="font-size: 0.7rem; letter-spacing: 0.5px;">Share via</div> <div class="hsm-share-grid"> <a id="share-twitter" class="btn btn-outline-secondary w-100" target="_blank" rel="noopener noreferrer"> Twitter </a> <a id="share-linkedin" class="btn btn-outline-secondary w-100" target="_blank" rel="noopener noreferrer"> LinkedIn </a> <a id="share-facebook" class="btn btn-outline-secondary w-100" target="_blank" rel="noopener noreferrer"> Facebook </a> </div> </div> </div> </div> <style> .heading-share-modal { position: fixed; inset: 0; display: flex; justify-content: center; align-items: center; background: rgba(0, 0, 0, 0.6); z-index: 1050; padding: 1rem; backdrop-filter: blur(4px); -webkit-backdrop-filter: blur(4px); } .heading-share-modal[hidden] { display: none !important; } .hsm-dialog { max-width: 420px; width: 100%; background: var(--bs-body-bg, #fff); color: var(--bs-body-color, #212529); border: 1px solid var(--bs-border-color, rgba(0,0,0,0.1)); border-radius: 1rem; box-shadow: 0 25px 50px -12px rgba(0, 0, 0, 0.25); overflow: hidden; animation: hsm-fade-in 0.2s ease-out; } @keyframes hsm-fade-in { from { opacity: 0; transform: scale(0.95); } to { opacity: 1; transform: scale(1); } } [data-bs-theme="dark"] .hsm-dialog { background: #1e293b; border-color: rgba(255,255,255,0.1); color: #f8f9fa; } .hsm-header { display: flex; justify-content: space-between; align-items: center; padding: 1rem 1.5rem; border-bottom: 1px solid var(--bs-border-color, rgba(0,0,0,0.1)); background: rgba(0,0,0,0.02); } [data-bs-theme="dark"] .hsm-header { background: rgba(255,255,255,0.02); border-color: rgba(255,255,255,0.1); } .hsm-close { background: transparent; border: none; color: inherit; opacity: 0.5; padding: 0.25rem 0.5rem; border-radius: 0.25rem; font-size: 1.2rem; line-height: 1; transition: opacity 0.2s; } .hsm-close:hover { opacity: 1; } .hsm-body { padding: 1.5rem; } .hsm-url-group { display: flex !important; align-items: stretch; } .hsm-url-group .form-control { flex: 1; min-width: 0; margin: 0; background: var(--bs-secondary-bg, #f8f9fa); border-color: var(--bs-border-color, #dee2e6); border-top-right-radius: 0; border-bottom-right-radius: 0; height: 42px; } .hsm-url-group .btn { flex: 0 0 auto; margin: 0; margin-left: -1px; border-top-left-radius: 0; border-bottom-left-radius: 0; height: 42px; display: flex; align-items: center; justify-content: center; padding: 0 1.25rem; z-index: 2; } [data-bs-theme="dark"] .hsm-url-group .form-control { background: #0f172a; border-color: #334155; color: #e2e8f0; } .hsm-share-grid { display: flex; flex-direction: column; gap: 0.5rem; } .hsm-share-grid .btn { display: flex; align-items: center; justify-content: center; font-size: 0.9rem; padding: 0.6rem; border-color: var(--bs-border-color); width: 100%; } [data-bs-theme="dark"] .hsm-share-grid .btn { color: #e2e8f0; border-color: #475569; } [data-bs-theme="dark"] .hsm-share-grid .btn:hover { background: #334155; border-color: #cbd5e1; } </style> <script> (function(){ const modal = document.getElementById('headingShareModal'); if(!modal) return; const input = modal.querySelector('#headingShareInput'); const copyBtn = modal.querySelector('.hsm-copy'); const twitter = modal.querySelector('#share-twitter'); const linkedin = modal.querySelector('#share-linkedin'); const facebook = modal.querySelector('#share-facebook'); const closeBtn = modal.querySelector('.hsm-close'); let lastFocus=null; let trapBound=false; function buildUrl(id){ return window.location.origin + window.location.pathname + '#' + id; } function isOpen(){ return !modal.hasAttribute('hidden'); } function hydrate(id){ const url=buildUrl(id); input.value=url; const enc=encodeURIComponent(url); const text=encodeURIComponent(document.title); if(twitter) twitter.href=`https://twitter.com/intent/tweet?url=${enc}&text=${text}`; if(linkedin) linkedin.href=`https://www.linkedin.com/sharing/share-offsite/?url=${enc}`; if(facebook) facebook.href=`https://www.facebook.com/sharer/sharer.php?u=${enc}`; } function openModal(id){ lastFocus=document.activeElement; hydrate(id); if(!isOpen()){ modal.removeAttribute('hidden'); } requestAnimationFrame(()=>{ input.focus(); }); trapFocus(); } function closeModal(){ if(!isOpen()) return; modal.setAttribute('hidden',''); if(lastFocus && typeof lastFocus.focus==='function') lastFocus.focus(); } function copyCurrent(){ try{ navigator.clipboard.writeText(input.value).then(()=>feedback(true),()=>fallback()); } catch(e){ fallback(); } } function fallback(){ input.select(); try{ document.execCommand('copy'); feedback(true);}catch(e){ feedback(false);} } function feedback(ok){ if(!copyBtn) return; const icon=copyBtn.querySelector('i'); if(!icon) return; const prev=copyBtn.getAttribute('data-prev')||icon.className; if(!copyBtn.getAttribute('data-prev')) copyBtn.setAttribute('data-prev',prev); icon.className= ok ? 'fa-duotone fa-clipboard-check':'fa-duotone fa-circle-exclamation'; setTimeout(()=>{ icon.className=prev; },1800); } function handleShareClick(e){ e.preventDefault(); const btn=e.currentTarget; const id=btn.getAttribute('data-share-target'); if(id) openModal(id); } function bindShareButtons(){ document.querySelectorAll('.h-share').forEach(btn=>{ if(!btn.dataset.hShareBound){ btn.addEventListener('click', handleShareClick); btn.dataset.hShareBound='1'; } }); } bindShareButtons(); if(document.readyState==='loading'){ document.addEventListener('DOMContentLoaded', bindShareButtons); } else { requestAnimationFrame(bindShareButtons); } document.addEventListener('click', function(e){ const shareBtn=e.target.closest && e.target.closest('.h-share'); if(shareBtn && !shareBtn.dataset.hShareBound){ handleShareClick.call(shareBtn, e); } }, true); document.addEventListener('click', e=>{ if(e.target===modal) closeModal(); if(e.target.closest && e.target.closest('.hsm-close')){ e.preventDefault(); closeModal(); } if(copyBtn && (e.target===copyBtn || (e.target.closest && e.target.closest('.hsm-copy')))) { e.preventDefault(); copyCurrent(); } }); document.addEventListener('keydown', e=>{ if(e.key==='Escape' && isOpen()) closeModal(); }); function trapFocus(){ if(trapBound) return; trapBound=true; modal.addEventListener('keydown', f=>{ if(f.key==='Tab' && isOpen()){ const focusable=[...modal.querySelectorAll('a[href],button,input,textarea,select,[tabindex]:not([tabindex="-1"])')].filter(el=>!el.hasAttribute('disabled')); if(!focusable.length) return; const first=focusable[0]; const last=focusable[focusable.length-1]; if(f.shiftKey && document.activeElement===first){ f.preventDefault(); last.focus(); } else if(!f.shiftKey && document.activeElement===last){ f.preventDefault(); first.focus(); } } }); } if(closeBtn) closeBtn.addEventListener('click', e=>{ e.preventDefault(); closeModal(); }); })(); </script>This guide covers benchmarking Geode for performance evaluation, capacity planning, and optimization validation. <h3 id="overview" class="position-relative d-flex align-items-center group"> Overview <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="overview" aria-haspopup="dialog" aria-label="Share link: Overview"> Share link </button> </h3>Effective benchmarking requires: <table> <thead> <tr> <th>Component</th> <th>Purpose</th> </tr> </thead> <tbody> <tr> <td>Clear objectives</td> <td>What are you measuring?</td> </tr> <tr> <td>Reproducible setup</td> <td>Consistent environment</td> </tr> <tr> <td>Realistic workloads</td> <td>Representative of production</td> </tr> <tr> <td>Proper metrics</td> <td>Latency, throughput, resource usage</td> </tr> <tr> <td>Statistical rigor</td> <td>Multiple runs, confidence intervals</td> </tr> </tbody> </table> <h3 id="benchmarking-objectives" class="position-relative d-flex align-items-center group"> Benchmarking Objectives <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="benchmarking-objectives" aria-haspopup="dialog" aria-label="Share link: Benchmarking Objectives"> Share link </button> </h3> <h4 id="common-objectives" class="position-relative d-flex align-items-center group"> Common Objectives <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="common-objectives" aria-haspopup="dialog" aria-label="Share link: Common Objectives"> Share link </button> </h4><table> <thead> <tr> <th>Objective</th> <th>Metrics</th> <th>Use Case</th> </tr> </thead> <tbody> <tr> <td>Baseline</td> <td>Throughput, latency</td> <td>Establish reference point</td> </tr> <tr> <td>Capacity planning</td> <td>Max throughput, breaking point</td> <td>Sizing infrastructure</td> </tr> <tr> <td>Regression testing</td> <td>Before/after comparison</td> <td>Validating changes</td> </tr> <tr> <td>Optimization</td> <td>Specific metric improvement</td> <td>Tuning configuration</td> </tr> <tr> <td>Competitive analysis</td> <td>Comparison with alternatives</td> <td>Technology selection</td> </tr> </tbody> </table> <h4 id="defining-success-criteria" class="position-relative d-flex align-items-center group"> Defining Success Criteria <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="defining-success-criteria" aria-haspopup="dialog" aria-label="Share link: Defining Success Criteria"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"># benchmark-criteria.yaml objectives: - name: "Query Latency" metric: "p99_latency_ms" target: "<100ms" - name: "Read Throughput" metric: "queries_per_second" target: ">10000" - name: "Write Throughput" metric: "writes_per_second" target: ">5000" - name: "Mixed Workload" metric: "operations_per_second" target: ">8000" workload: "50% read, 50% write" </code></pre></div> <h3 id="environment-setup" class="position-relative d-flex align-items-center group"> Environment Setup <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="environment-setup" aria-haspopup="dialog" aria-label="Share link: Environment Setup"> Share link </button> </h3> <h4 id="hardware-requirements" class="position-relative d-flex align-items-center group"> Hardware Requirements <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="hardware-requirements" aria-haspopup="dialog" aria-label="Share link: Hardware Requirements"> Share link </button> </h4>Benchmark Server: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"># Recommended for benchmarking cpu: 16+ cores (same SKU as production) memory: 64GB+ RAM storage: NVMe SSD (1TB+) network: 10Gbps+ </code></pre></div>Load Generator: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"># Separate machine for load generation cpu: 8+ cores memory: 16GB+ RAM network: 10Gbps+ (same network as server) </code></pre></div> <h4 id="software-configuration" class="position-relative d-flex align-items-center group"> Software Configuration <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="software-configuration" aria-haspopup="dialog" aria-label="Share link: Software Configuration"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"># geode.yaml - Benchmark configuration server: listen: '0.0.0.0:3141' max_connections: 50000 storage: page_cache_size: '32GB' # 50% of RAM page_size: 8192 wal_sync_interval: 100ms query: max_concurrent_queries: 1000 query_timeout: 30s query_memory_limit: '2GB' # Disable features that add overhead logging: level: warn # Reduce logging slow_query: enabled: false monitoring: detailed_metrics: false # Reduce metric overhead </code></pre></div> <h4 id="system-tuning" class="position-relative d-flex align-items-center group"> System Tuning <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="system-tuning" aria-haspopup="dialog" aria-label="Share link: System Tuning"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash">#!/bin/bash # system-tuning.sh - Prepare system for benchmarking # Increase file descriptors ulimit -n 1000000 # Tune TCP settings sysctl -w net.core.somaxconn=65535 sysctl -w net.ipv4.tcp_max_syn_backlog=65535 sysctl -w net.core.netdev_max_backlog=65535 # Disable swap swapoff -a # Set CPU governor to performance for cpu in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do echo performance > $cpu done # Disable transparent huge pages echo never > /sys/kernel/mm/transparent_hugepage/enabled </code></pre></div> <h3 id="benchmark-tools" class="position-relative d-flex align-items-center group"> Benchmark Tools <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="benchmark-tools" aria-haspopup="dialog" aria-label="Share link: Benchmark Tools"> Share link </button> </h3> <h4 id="geode-benchmark-tool" class="position-relative d-flex align-items-center group"> Geode Benchmark Tool <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="geode-benchmark-tool" aria-haspopup="dialog" aria-label="Share link: Geode Benchmark Tool"> Share link </button> </h4>Built-in benchmarking utility: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"># Basic benchmark geode benchmark \ --host localhost:3141 \ --duration 60s \ --threads 16 # Custom workload geode benchmark \ --host localhost:3141 \ --workload mixed \ --read-percent 80 \ --write-percent 20 \ --duration 300s \ --threads 32 \ --connections 100 # Output: # ============================================ # Geode Benchmark Results # ============================================ # Duration: 300s # Threads: 32 # Connections: 100 # # Throughput: # Total Operations: 2,450,000 # Operations/sec: 8,166 # Read ops/sec: 6,533 # Write ops/sec: 1,633 # # Latency (ms): # Min: 0.2 # Max: 45.3 # Mean: 2.1 # P50: 1.8 # P95: 4.5 # P99: 8.2 # P99.9: 15.3 # # Errors: 0 (0.00%) </code></pre></div> <h4 id="custom-benchmark-scripts" class="position-relative d-flex align-items-center group"> Custom Benchmark Scripts <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="custom-benchmark-scripts" aria-haspopup="dialog" aria-label="Share link: Custom Benchmark Scripts"> Share link </button> </h4>Python Benchmark: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python">#!/usr/bin/env python3 # benchmark.py import asyncio import time import statistics from geode_client import GeodeClient async def benchmark_reads(client, num_queries, concurrency): """Benchmark read queries""" latencies = [] errors = 0 async def run_query(): nonlocal errors start = time.perf_counter() try: await client.query("MATCH (p:Person) RETURN p LIMIT 10") latencies.append((time.perf_counter() - start) * 1000) except Exception: errors += 1 # Run concurrent queries tasks = [] for _ in range(num_queries): if len(tasks) >= concurrency: done, tasks = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED) tasks = list(tasks) tasks.append(asyncio.create_task(run_query())) await asyncio.gather(*tasks) return { 'count': num_queries, 'errors': errors, 'latencies': latencies, 'p50': statistics.median(latencies), 'p95': statistics.quantiles(latencies, n=20)[18], 'p99': statistics.quantiles(latencies, n=100)[98], 'mean': statistics.mean(latencies), 'qps': num_queries / (sum(latencies) / 1000 / concurrency) } async def main(): client = await GeodeClient.connect("localhost:3141") # Warmup print("Warming up...") await benchmark_reads(client, 1000, 10) # Benchmark print("Running benchmark...") results = await benchmark_reads(client, 100000, 100) print(f""" Benchmark Results ================= Queries: {results['count']} Errors: {results['errors']} QPS: {results['qps']:.0f} Latency (ms): Mean: {results['mean']:.2f} P50: {results['p50']:.2f} P95: {results['p95']:.2f} P99: {results['p99']:.2f} """) if __name__ == "__main__": asyncio.run(main()) </code></pre></div>Go Benchmark: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go">// benchmark_test.go package main import ( "context" "sync" "testing" "time" "go.codepros.org/geode" ) func BenchmarkReadQuery(b *testing.B) { ctx := context.Background() client, _ := geode.Connect(ctx, "localhost:3141") defer client.Close() b.ResetTimer() b.RunParallel(func(pb *testing.PB) { for pb.Next() { _, _ = client.Query(ctx, "MATCH (p:Person) RETURN p LIMIT 10") } }) } func BenchmarkWriteQuery(b *testing.B) { ctx := context.Background() client, _ := geode.Connect(ctx, "localhost:3141") defer client.Close() b.ResetTimer() for i := 0; i < b.N; i++ { _, _ = client.Query(ctx, "CREATE (p:Person {id: $id, name: 'Test'})", map[string]interface{}{"id": i}) } } // Run: go test -bench=. -benchtime=60s -cpu=1,4,8,16 </code></pre></div> <h4 id="load-generation-tools" class="position-relative d-flex align-items-center group"> Load Generation Tools <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="load-generation-tools" aria-haspopup="dialog" aria-label="Share link: Load Generation Tools"> Share link </button> </h4>wrk2: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"># Install wrk2 git clone https://github.com/giltene/wrk2.git cd wrk2 && make # Run constant-rate benchmark ./wrk -t4 -c100 -d60s -R10000 \ --latency \ -s benchmark.lua \ http://localhost:8080/query </code></pre></div>hey: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"># HTTP-based benchmark hey -n 100000 -c 100 -m POST \ -H "Content-Type: application/json" \ -d '{"query": "MATCH (n) RETURN n LIMIT 10"}' \ http://localhost:8080/query </code></pre></div> <h3 id="workload-types" class="position-relative d-flex align-items-center group"> Workload Types <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="workload-types" aria-haspopup="dialog" aria-label="Share link: Workload Types"> Share link </button> </h3> <h4 id="read-heavy-workload" class="position-relative d-flex align-items-center group"> Read-Heavy Workload <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="read-heavy-workload" aria-haspopup="dialog" aria-label="Share link: Read-Heavy Workload"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"># 95% reads, 5% writes geode benchmark \ --workload custom \ --read-percent 95 \ --write-percent 5 \ --queries-file read-queries.gql # read-queries.gql MATCH (p:Person {email: $email}) RETURN p; MATCH (p:Person)-[:KNOWS]->(f) WHERE p.id = $id RETURN f; MATCH (p:Person) WHERE p.age > $min AND p.age < $max RETURN p LIMIT 100; </code></pre></div> <h4 id="write-heavy-workload" class="position-relative d-flex align-items-center group"> Write-Heavy Workload <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="write-heavy-workload" aria-haspopup="dialog" aria-label="Share link: Write-Heavy Workload"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"># 20% reads, 80% writes geode benchmark \ --workload custom \ --read-percent 20 \ --write-percent 80 \ --queries-file write-queries.gql # write-queries.gql CREATE (p:Person {id: $id, name: $name, email: $email}); MATCH (a:Person {id: $from}), (b:Person {id: $to}) CREATE (a)-[:KNOWS]->(b); MATCH (p:Person {id: $id}) SET p.updated = datetime(); </code></pre></div> <h4 id="mixed-oltp-workload" class="position-relative d-flex align-items-center group"> Mixed OLTP Workload <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="mixed-oltp-workload" aria-haspopup="dialog" aria-label="Share link: Mixed OLTP Workload"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"># Balanced OLTP workload geode benchmark \ --workload oltp \ --duration 300s \ --threads 64 # OLTP workload includes: # - 50% point reads # - 20% range reads # - 15% inserts # - 10% updates # - 5% deletes </code></pre></div> <h4 id="analytical-workload" class="position-relative d-flex align-items-center group"> Analytical Workload <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="analytical-workload" aria-haspopup="dialog" aria-label="Share link: Analytical Workload"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"># Analytical queries geode benchmark \ --workload analytical \ --queries-file analytical-queries.gql # analytical-queries.gql MATCH (p:Person)-[:KNOWS]->(f) RETURN p.city, count(f) AS friends GROUP BY p.city ORDER BY friends DESC LIMIT 10; MATCH path = shortestPath((a:Person)-[:KNOWS*1..6]->(b:Person)) WHERE a.id = $start AND b.id = $end RETURN path; </code></pre></div> <h4 id="graph-traversal-workload" class="position-relative d-flex align-items-center group"> Graph Traversal Workload <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="graph-traversal-workload" aria-haspopup="dialog" aria-label="Share link: Graph Traversal Workload"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"># Deep graph traversals geode benchmark \ --workload traversal \ --max-depth 5 \ --queries-file traversal-queries.gql # traversal-queries.gql MATCH (a:Person {id: $id})-[:KNOWS*1..3]->(b) RETURN DISTINCT b; MATCH (a:Person)-[:KNOWS*2]->(b:Person)-[:KNOWS]->(c:Person) WHERE a.id = $id RETURN c LIMIT 100; </code></pre></div> <h3 id="data-generation" class="position-relative d-flex align-items-center group"> Data Generation <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="data-generation" aria-haspopup="dialog" aria-label="Share link: Data Generation"> Share link </button> </h3> <h4 id="synthetic-data-generator" class="position-relative d-flex align-items-center group"> Synthetic Data Generator <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="synthetic-data-generator" aria-haspopup="dialog" aria-label="Share link: Synthetic Data Generator"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"># Generate test data geode data-gen \ --nodes 1000000 \ --edges 5000000 \ --node-labels Person,Company,Product \ --edge-types KNOWS,WORKS_AT,PURCHASED \ --output test-data/ # Load generated data geode import \ --source test-data/ \ --format geode-export </code></pre></div> <h4 id="custom-data-generator" class="position-relative d-flex align-items-center group"> Custom Data Generator <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="custom-data-generator" aria-haspopup="dialog" aria-label="Share link: Custom Data Generator"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python">#!/usr/bin/env python3 # generate_data.py import random import json from faker import Faker fake = Faker() def generate_persons(n): """Generate person nodes""" for i in range(n): yield { "id": i, "name": fake.name(), "email": fake.email(), "age": random.randint(18, 80), "city": fake.city(), "created_at": fake.date_time_this_decade().isoformat() } def generate_knows_relationships(n_persons, avg_friends): """Generate KNOWS relationships""" for person_id in range(n_persons): n_friends = random.randint(0, avg_friends * 2) friends = random.sample(range(n_persons), min(n_friends, n_persons - 1)) for friend_id in friends: if friend_id != person_id: yield { "source": person_id, "target": friend_id, "since": fake.date_this_decade().isoformat() } # Generate data N_PERSONS = 1000000 AVG_FRIENDS = 10 with open('persons.jsonl', 'w') as f: for person in generate_persons(N_PERSONS): f.write(json.dumps(person) + '\n') with open('knows.jsonl', 'w') as f: for rel in generate_knows_relationships(N_PERSONS, AVG_FRIENDS): f.write(json.dumps(rel) + '\n') </code></pre></div> <h3 id="running-benchmarks" class="position-relative d-flex align-items-center group"> Running Benchmarks <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="running-benchmarks" aria-haspopup="dialog" aria-label="Share link: Running Benchmarks"> Share link </button> </h3> <h4 id="benchmark-script" class="position-relative d-flex align-items-center group"> Benchmark Script <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="benchmark-script" aria-haspopup="dialog" aria-label="Share link: Benchmark Script"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash">#!/bin/bash # run-benchmark.sh set -euo pipefail RESULTS_DIR="results/$(date +%Y%m%d-%H%M%S)" mkdir -p "$RESULTS_DIR" # Configuration DURATION=300 THREADS="1 4 8 16 32 64" WORKLOADS="read-heavy write-heavy mixed" # System info echo "Collecting system info..." uname -a > "$RESULTS_DIR/system-info.txt" lscpu >> "$RESULTS_DIR/system-info.txt" free -h >> "$RESULTS_DIR/system-info.txt" geode --version >> "$RESULTS_DIR/system-info.txt" # Warmup echo "Warming up..." geode benchmark --duration 60s --threads 16 > /dev/null # Run benchmarks for workload in $WORKLOADS; do for threads in $THREADS; do echo "Running: workload=$workload threads=$threads" OUTPUT_FILE="$RESULTS_DIR/${workload}_${threads}threads.json" geode benchmark \ --workload $workload \ --duration $DURATION \ --threads $threads \ --output json \ > "$OUTPUT_FILE" # Cool down between runs sleep 30 done done # Generate summary echo "Generating summary..." python3 summarize_results.py "$RESULTS_DIR" > "$RESULTS_DIR/summary.md" echo "Results saved to $RESULTS_DIR" </code></pre></div> <h4 id="continuous-benchmarking" class="position-relative d-flex align-items-center group"> Continuous Benchmarking <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="continuous-benchmarking" aria-haspopup="dialog" aria-label="Share link: Continuous Benchmarking"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"># .github/workflows/benchmark.yml name: Performance Benchmark on: push: branches: [main] schedule: - cron: '0 0 * * 0' # Weekly jobs: benchmark: runs-on: self-hosted # Dedicated benchmark runner steps: - uses: actions/checkout@v3 - name: Build Geode run: make release - name: Start Geode run: ./zig-out/bin/geode serve & - name: Load test data run: ./scripts/load-benchmark-data.sh - name: Run benchmarks run: ./scripts/run-benchmark.sh - name: Upload results uses: actions/upload-artifact@v3 with: name: benchmark-results path: results/ - name: Compare with baseline run: | python3 scripts/compare_benchmark.py \ --baseline results/baseline.json \ --current results/latest.json \ --threshold 5% # Fail if >5% regression </code></pre></div> <h3 id="analyzing-results" class="position-relative d-flex align-items-center group"> Analyzing Results <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="analyzing-results" aria-haspopup="dialog" aria-label="Share link: Analyzing Results"> Share link </button> </h3> <h4 id="metrics-to-analyze" class="position-relative d-flex align-items-center group"> Metrics to Analyze <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="metrics-to-analyze" aria-haspopup="dialog" aria-label="Share link: Metrics to Analyze"> Share link </button> </h4><table> <thead> <tr> <th>Metric</th> <th>What It Tells You</th> </tr> </thead> <tbody> <tr> <td>Throughput (ops/sec)</td> <td>System capacity</td> </tr> <tr> <td>Latency (p50, p95, p99)</td> <td>Response time distribution</td> </tr> <tr> <td>Error rate</td> <td>Reliability under load</td> </tr> <tr> <td>CPU utilization</td> <td>Compute efficiency</td> </tr> <tr> <td>Memory usage</td> <td>Memory efficiency</td> </tr> <tr> <td>Disk I/O</td> <td>Storage bottlenecks</td> </tr> <tr> <td>Network I/O</td> <td>Network bottlenecks</td> </tr> </tbody> </table> <h4 id="result-analysis-script" class="position-relative d-flex align-items-center group"> Result Analysis Script <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="result-analysis-script" aria-haspopup="dialog" aria-label="Share link: Result Analysis Script"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python">#!/usr/bin/env python3 # analyze_results.py import json import pandas as pd import matplotlib.pyplot as plt def load_results(results_dir): """Load benchmark results""" results = [] for file in Path(results_dir).glob("*.json"): with open(file) as f: data = json.load(f) data['file'] = file.name results.append(data) return pd.DataFrame(results) def analyze_throughput(df): """Analyze throughput scaling""" fig, ax = plt.subplots() for workload in df['workload'].unique(): subset = df[df['workload'] == workload] ax.plot(subset['threads'], subset['throughput'], marker='o', label=workload) ax.set_xlabel('Threads') ax.set_ylabel('Throughput (ops/sec)') ax.set_title('Throughput vs Concurrency') ax.legend() ax.grid(True) plt.savefig('throughput_scaling.png') def analyze_latency(df): """Analyze latency distribution""" fig, axes = plt.subplots(1, 3, figsize=(15, 5)) for i, percentile in enumerate(['p50', 'p95', 'p99']): ax = axes[i] for workload in df['workload'].unique(): subset = df[df['workload'] == workload] ax.plot(subset['threads'], subset[percentile], marker='o', label=workload) ax.set_xlabel('Threads') ax.set_ylabel(f'{percentile} Latency (ms)') ax.set_title(f'{percentile} Latency vs Concurrency') ax.legend() ax.grid(True) plt.tight_layout() plt.savefig('latency_analysis.png') def generate_report(df): """Generate markdown report""" report = """# Benchmark Report ## Summary | Metric | Value | |--------|-------| | Max Throughput | {max_throughput:.0f} ops/sec | | Best P99 Latency | {best_p99:.2f} ms | | Optimal Threads | {optimal_threads} | ## Detailed Results {detailed_table} """ max_throughput = df['throughput'].max() best_p99 = df['p99'].min() optimal_threads = df.loc[df['throughput'].idxmax(), 'threads'] detailed_table = df.to_markdown() return report.format( max_throughput=max_throughput, best_p99=best_p99, optimal_threads=optimal_threads, detailed_table=detailed_table ) if __name__ == "__main__": import sys results_dir = sys.argv[1] df = load_results(results_dir) analyze_throughput(df) analyze_latency(df) print(generate_report(df)) </code></pre></div> <h4 id="comparison-report" class="position-relative d-flex align-items-center group"> Comparison Report <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="comparison-report" aria-haspopup="dialog" aria-label="Share link: Comparison Report"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-markdown" data-lang="markdown"># Performance Comparison Report ## Configuration - **Baseline**: v0.1.2 - **Current**: v0.2.18 - **Hardware**: 16 cores, 64GB RAM, NVMe SSD - **Data**: 1M nodes, 10M relationships ## Results | Metric | Baseline | Current | Change | |--------|----------|---------|--------| | Read QPS | 15,234 | 16,891 | +10.9% | | Write QPS | 5,123 | 5,456 | +6.5% | | P50 Latency | 1.2ms | 1.1ms | -8.3% | | P99 Latency | 8.5ms | 7.2ms | -15.3% | | Peak Memory | 12GB | 11GB | -8.3% | ## Conclusion Version 0.2.18 shows improvements across all metrics, with particularly notable improvements in P99 latency (-15.3%). No regressions detected. </code></pre></div> <h3 id="best-practices" class="position-relative d-flex align-items-center group"> Best Practices <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="best-practices" aria-haspopup="dialog" aria-label="Share link: Best Practices"> Share link </button> </h3> <h4 id="benchmarking-best-practices" class="position-relative d-flex align-items-center group"> Benchmarking Best Practices <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="benchmarking-best-practices" aria-haspopup="dialog" aria-label="Share link: Benchmarking Best Practices"> Share link </button> </h4><ol> <li>Isolate the system: No other workloads during benchmark</li> <li>Warm up: Run warmup phase before measurements</li> <li>Multiple runs: At least 3 runs for statistical validity</li> <li>Realistic data: Use production-like data size and distribution</li> <li>Monitor resources: Track CPU, memory, disk, network</li> <li>Document everything: Configuration, versions, hardware</li> </ol> <h4 id="common-pitfalls" class="position-relative d-flex align-items-center group"> Common Pitfalls <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="common-pitfalls" aria-haspopup="dialog" aria-label="Share link: Common Pitfalls"> Share link </button> </h4><ol> <li>Insufficient warmup: JIT compilation, cache warming</li> <li>Coordinated omission: Load generator waiting skews latency</li> <li>Client bottleneck: Load generator limiting throughput</li> <li>Network effects: Localhost vs network overhead</li> <li>Small data sets: Cache effects masking real performance</li> <li>Single run: Statistical noise in results</li> </ol> <h4 id="reporting-guidelines" class="position-relative d-flex align-items-center group"> Reporting Guidelines <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="reporting-guidelines" aria-haspopup="dialog" aria-label="Share link: Reporting Guidelines"> Share link </button> </h4><ol> <li>Include configuration: Hardware, software versions, settings</li> <li>Show distribution: Not just averages, show percentiles</li> <li>Multiple metrics: Throughput AND latency</li> <li>Error rates: Include failures in results</li> <li>Reproducibility: Share scripts and data</li> </ol> <h3 id="related-documentation" class="position-relative d-flex align-items-center group"> Related Documentation <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="related-documentation" aria-haspopup="dialog" aria-label="Share link: Related Documentation"> Share link </button> </h3><ul> <li><a href="/docs/performance/query-optimization/" >Query Optimization</a> - Optimize queries</li> <li><a href="/docs/performance/" >Performance Tuning</a> - System tuning</li> <li><a href="/docs/operations/monitoring/" >Monitoring</a> - Production monitoring</li> <li><a href="/docs/deployment/deployment-patterns/" >Hardware Recommendations</a> - Hardware sizing</li> </ul>