Continuous Integration

<h2 id="continuous-integration-for-geode" class="position-relative d-flex align-items-center group"> Continuous Integration for Geode <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="continuous-integration-for-geode" aria-haspopup="dialog" aria-label="Share link: Continuous Integration for Geode"> Share link </button> </h2><div id="headingShareModal" class="heading-share-modal" role="dialog" aria-modal="true" aria-labelledby="headingShareTitle" hidden> <div class="hsm-dialog" role="document"> <div class="hsm-header"> <h2 id="headingShareTitle" class="h6 mb-0 fw-bold">Share this section</h2> <button type="button" class="hsm-close" aria-label="Close"> </button> </div> <div class="hsm-body"> <label for="headingShareInput" class="form-label small text-muted mb-1 text-uppercase fw-bold" style="font-size: 0.7rem; letter-spacing: 0.5px;">Permalink</label> <div class="input-group mb-4 hsm-url-group"> <input id="headingShareInput" type="text" class="form-control font-monospace" readonly aria-readonly="true" style="font-size: 0.85rem;" /> <button class="btn btn-primary hsm-copy" type="button" aria-label="Copy" title="Copy"> </button> </div> <div class="small fw-bold mb-2 text-muted text-uppercase" style="font-size: 0.7rem; letter-spacing: 0.5px;">Share via</div> <div class="hsm-share-grid"> <a id="share-twitter" class="btn btn-outline-secondary w-100" target="_blank" rel="noopener noreferrer"> Twitter </a> <a id="share-linkedin" class="btn btn-outline-secondary w-100" target="_blank" rel="noopener noreferrer"> LinkedIn </a> <a id="share-facebook" class="btn btn-outline-secondary w-100" target="_blank" rel="noopener noreferrer"> Facebook </a> </div> </div> </div> </div> <style> .heading-share-modal { position: fixed; inset: 0; display: flex; justify-content: center; align-items: center; background: rgba(0, 0, 0, 0.6); z-index: 1050; padding: 1rem; backdrop-filter: blur(4px); -webkit-backdrop-filter: blur(4px); } .heading-share-modal[hidden] { display: none !important; } .hsm-dialog { max-width: 420px; width: 100%; background: var(--bs-body-bg, #fff); color: var(--bs-body-color, #212529); border: 1px solid var(--bs-border-color, rgba(0,0,0,0.1)); border-radius: 1rem; box-shadow: 0 25px 50px -12px rgba(0, 0, 0, 0.25); overflow: hidden; animation: hsm-fade-in 0.2s ease-out; } @keyframes hsm-fade-in { from { opacity: 0; transform: scale(0.95); } to { opacity: 1; transform: scale(1); } } [data-bs-theme="dark"] .hsm-dialog { background: #1e293b; border-color: rgba(255,255,255,0.1); color: #f8f9fa; } .hsm-header { display: flex; justify-content: space-between; align-items: center; padding: 1rem 1.5rem; border-bottom: 1px solid var(--bs-border-color, rgba(0,0,0,0.1)); background: rgba(0,0,0,0.02); } [data-bs-theme="dark"] .hsm-header { background: rgba(255,255,255,0.02); border-color: rgba(255,255,255,0.1); } .hsm-close { background: transparent; border: none; color: inherit; opacity: 0.5; padding: 0.25rem 0.5rem; border-radius: 0.25rem; font-size: 1.2rem; line-height: 1; transition: opacity 0.2s; } .hsm-close:hover { opacity: 1; } .hsm-body { padding: 1.5rem; } .hsm-url-group { display: flex !important; align-items: stretch; } .hsm-url-group .form-control { flex: 1; min-width: 0; margin: 0; background: var(--bs-secondary-bg, #f8f9fa); border-color: var(--bs-border-color, #dee2e6); border-top-right-radius: 0; border-bottom-right-radius: 0; height: 42px; } .hsm-url-group .btn { flex: 0 0 auto; margin: 0; margin-left: -1px; border-top-left-radius: 0; border-bottom-left-radius: 0; height: 42px; display: flex; align-items: center; justify-content: center; padding: 0 1.25rem; z-index: 2; } [data-bs-theme="dark"] .hsm-url-group .form-control { background: #0f172a; border-color: #334155; color: #e2e8f0; } .hsm-share-grid { display: flex; flex-direction: column; gap: 0.5rem; } .hsm-share-grid .btn { display: flex; align-items: center; justify-content: center; font-size: 0.9rem; padding: 0.6rem; border-color: var(--bs-border-color); width: 100%; } [data-bs-theme="dark"] .hsm-share-grid .btn { color: #e2e8f0; border-color: #475569; } [data-bs-theme="dark"] .hsm-share-grid .btn:hover { background: #334155; border-color: #cbd5e1; } </style> <script> (function(){ const modal = document.getElementById('headingShareModal'); if(!modal) return; const input = modal.querySelector('#headingShareInput'); const copyBtn = modal.querySelector('.hsm-copy'); const twitter = modal.querySelector('#share-twitter'); const linkedin = modal.querySelector('#share-linkedin'); const facebook = modal.querySelector('#share-facebook'); const closeBtn = modal.querySelector('.hsm-close'); let lastFocus=null; let trapBound=false; function buildUrl(id){ return window.location.origin + window.location.pathname + '#' + id; } function isOpen(){ return !modal.hasAttribute('hidden'); } function hydrate(id){ const url=buildUrl(id); input.value=url; const enc=encodeURIComponent(url); const text=encodeURIComponent(document.title); if(twitter) twitter.href=`https://twitter.com/intent/tweet?url=${enc}&text=${text}`; if(linkedin) linkedin.href=`https://www.linkedin.com/sharing/share-offsite/?url=${enc}`; if(facebook) facebook.href=`https://www.facebook.com/sharer/sharer.php?u=${enc}`; } function openModal(id){ lastFocus=document.activeElement; hydrate(id); if(!isOpen()){ modal.removeAttribute('hidden'); } requestAnimationFrame(()=>{ input.focus(); }); trapFocus(); } function closeModal(){ if(!isOpen()) return; modal.setAttribute('hidden',''); if(lastFocus && typeof lastFocus.focus==='function') lastFocus.focus(); } function copyCurrent(){ try{ navigator.clipboard.writeText(input.value).then(()=>feedback(true),()=>fallback()); } catch(e){ fallback(); } } function fallback(){ input.select(); try{ document.execCommand('copy'); feedback(true);}catch(e){ feedback(false);} } function feedback(ok){ if(!copyBtn) return; const icon=copyBtn.querySelector('i'); if(!icon) return; const prev=copyBtn.getAttribute('data-prev')||icon.className; if(!copyBtn.getAttribute('data-prev')) copyBtn.setAttribute('data-prev',prev); icon.className= ok ? 'fa-duotone fa-clipboard-check':'fa-duotone fa-circle-exclamation'; setTimeout(()=>{ icon.className=prev; },1800); } function handleShareClick(e){ e.preventDefault(); const btn=e.currentTarget; const id=btn.getAttribute('data-share-target'); if(id) openModal(id); } function bindShareButtons(){ document.querySelectorAll('.h-share').forEach(btn=>{ if(!btn.dataset.hShareBound){ btn.addEventListener('click', handleShareClick); btn.dataset.hShareBound='1'; } }); } bindShareButtons(); if(document.readyState==='loading'){ document.addEventListener('DOMContentLoaded', bindShareButtons); } else { requestAnimationFrame(bindShareButtons); } document.addEventListener('click', function(e){ const shareBtn=e.target.closest && e.target.closest('.h-share'); if(shareBtn && !shareBtn.dataset.hShareBound){ handleShareClick.call(shareBtn, e); } }, true); document.addEventListener('click', e=>{ if(e.target===modal) closeModal(); if(e.target.closest && e.target.closest('.hsm-close')){ e.preventDefault(); closeModal(); } if(copyBtn && (e.target===copyBtn || (e.target.closest && e.target.closest('.hsm-copy')))) { e.preventDefault(); copyCurrent(); } }); document.addEventListener('keydown', e=>{ if(e.key==='Escape' && isOpen()) closeModal(); }); function trapFocus(){ if(trapBound) return; trapBound=true; modal.addEventListener('keydown', f=>{ if(f.key==='Tab' && isOpen()){ const focusable=[...modal.querySelectorAll('a[href],button,input,textarea,select,[tabindex]:not([tabindex="-1"])')].filter(el=>!el.hasAttribute('disabled')); if(!focusable.length) return; const first=focusable[0]; const last=focusable[focusable.length-1]; if(f.shiftKey && document.activeElement===first){ f.preventDefault(); last.focus(); } else if(!f.shiftKey && document.activeElement===last){ f.preventDefault(); first.focus(); } } }); } if(closeBtn) closeBtn.addEventListener('click', e=>{ e.preventDefault(); closeModal(); }); })(); </script>Continuous Integration (CI) is the practice of automatically building, testing, and validating code changes as they’re committed to version control. For Geode, CI ensures that every change maintains database correctness, preserves GQL standards compliance, prevents performance regressions, and validates polyglot client compatibility. <h3 id="foundations-of-continuous-integration" class="position-relative d-flex align-items-center group"> Foundations of Continuous Integration <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="foundations-of-continuous-integration" aria-haspopup="dialog" aria-label="Share link: Foundations of Continuous Integration"> Share link </button> </h3>Continuous Integration is built on several core principles: Frequent Integration: Developers integrate code changes multiple times daily, preventing large merge conflicts and integration issues. Automated Building: Every commit triggers an automated build process that compiles the codebase and identifies compilation errors immediately. Automated Testing: Comprehensive test suites run automatically, providing rapid feedback on functionality and preventing regressions. Fast Feedback: Developers receive build and test results within minutes, enabling quick fixes before context switching. Maintain a Single Source Repository: All code, tests, and configuration reside in version control (Git), ensuring reproducibility. For database systems like Geode, CI is particularly critical because bugs can result in data corruption, lost transactions, or query correctness issues that are difficult to debug in production. <h3 id="geodes-ci-implementation" class="position-relative d-flex align-items-center group"> Geode&rsquo;s CI Implementation <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="geodes-ci-implementation" aria-haspopup="dialog" aria-label="Share link: Geodes CI Implementation"> Share link </button> </h3> <h4 id="repository-structure" class="position-relative d-flex align-items-center group"> Repository Structure <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="repository-structure" aria-haspopup="dialog" aria-label="Share link: Repository Structure"> Share link </button> </h4>Geode’s monorepo structure enables comprehensive CI across all components: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback">/home/benji/src/gl/devnw/codepros/geode/ ├── geode/ # Server (Zig) │ ├── src/ │ ├── test/ │ └── build.zig ├── geode-client-go/ # Go client ├── geode-client-python/ # Python client ├── geode-client-rust/ # Rust client ├── geode-client-zig/ # Zig client ├── geode-test-harness/ # Cross-client tests └── .github/workflows/ # CI configuration </code></pre></div>This structure allows CI to: <ul> <li>Build all components in a single pipeline</li> <li>Test server changes against all clients</li> <li>Validate cross-client compatibility</li> <li>Ensure end-to-end functionality</li> </ul> <h4 id="build-process" class="position-relative d-flex align-items-center group"> Build Process <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="build-process" aria-haspopup="dialog" aria-label="Share link: Build Process"> Share link </button> </h4>Server Build (Zig): <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"># geode/Makefile build: zig build -Doptimize=ReleaseSafe test: zig build test geodetestlab-comprehensive: zig build test # Run comprehensive test suite (1,688 tests) ./zig-out/bin/geodetestlab --comprehensive </code></pre></div>CI Build Configuration: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"># .github/workflows/build.yml name: Build Geode on: push: branches: [main, develop] pull_request: branches: [main] jobs: build-zig: name: Build Server (Zig ${{ matrix.zig }}) runs-on: ${{ matrix.os }} strategy: matrix: os: [ubuntu-24.04, macos-14] zig: [0.1.0] steps: - uses: actions/checkout@v4 - name: Setup Zig uses: goto-bus-stop/setup-zig@v2 with: version: ${{ matrix.zig }} - name: Build Geode working-directory: geode run: | zig build -Doptimize=ReleaseSafe ./zig-out/bin/geode --version - name: Verify Binary run: | file geode/zig-out/bin/geode ldd geode/zig-out/bin/geode || true - name: Upload Binary uses: actions/upload-artifact@v4 with: name: geode-${{ matrix.os }}-${{ matrix.zig }} path: geode/zig-out/bin/geode retention-days: 7 </code></pre></div> <h4 id="automated-testing" class="position-relative d-flex align-items-center group"> Automated Testing <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="automated-testing" aria-haspopup="dialog" aria-label="Share link: Automated Testing"> Share link </button> </h4>Geode’s CI runs multiple test suites: 1. Unit Tests (Core functionality): <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">unit-tests: name: Unit Tests runs-on: ubuntu-24.04 needs: build-zig steps: - uses: actions/checkout@v4 - name: Setup Zig uses: goto-bus-stop/setup-zig@v2 with: version: 0.1.0 - name: Run Unit Tests working-directory: geode run: | zig build test -Dtest-filter="*" - name: Parse Test Results run: | # Extract test counts echo "Tests run, passed, failed" </code></pre></div>2. GeodeTestLab Comprehensive Suite (1,688 tests): <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"> - name: Run GeodeTestLab working-directory: geode run: | make geodetestlab-comprehensive - name: Check Pass Rate run: | # Require 97.4% pass rate (1644/1688) python scripts/check-pass-rate.py \ --results geode/test-results.json \ --threshold 0.974 </code></pre></div>3. GQL Compliance Tests (ISO/IEC 39075:2024): <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">gql-compliance: name: GQL Compliance needs: build-zig steps: - name: Download Server Binary uses: actions/download-artifact@v4 - name: Start Geode run: | chmod +x geode ./geode serve --listen 0.0.0.0:3141 & sleep 5 - name: Run Conformance Profile Tests run: | # Must pass profile scope (see conformance profile) python gql/iso-tests/run-suite.py \ --host localhost:3141 \ --strict - name: Enforce Conformance Profile run: | if [ $? -ne 0 ]; then echo "GQL conformance profile tests failed - blocking merge" exit 1 fi </code></pre></div>4. Client Library Tests: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">client-tests: name: Test ${{ matrix.client }} Client needs: build-zig strategy: matrix: client: [go, python, rust, zig] steps: - name: Download Server uses: actions/download-artifact@v4 - name: Start Server run: | ./geode serve & sleep 5 - name: Setup ${{ matrix.client }} uses: ./.github/actions/setup-${{ matrix.client }} - name: Run Tests working-directory: geode-client-${{ matrix.client }} run: make test - name: Upload Results if: always() uses: actions/upload-artifact@v4 with: name: test-results-${{ matrix.client }} path: geode-client-${{ matrix.client }}/test-results/ </code></pre></div>5. Cross-Client Integration Tests: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">integration-tests: name: Integration Tests needs: client-tests steps: - name: Setup Test Harness working-directory: geode-test-harness run: make setup - name: Run All Client Tests run: | make test-all make test-all-html - name: Upload HTML Report uses: actions/upload-artifact@v4 with: name: integration-report path: geode-test-harness/reports/ </code></pre></div> <h4 id="code-quality-checks" class="position-relative d-flex align-items-center group"> Code Quality Checks <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="code-quality-checks" aria-haspopup="dialog" aria-label="Share link: Code Quality Checks"> Share link </button> </h4>Linting and Formatting: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">code-quality: name: Code Quality runs-on: ubuntu-24.04 steps: - uses: actions/checkout@v4 - name: Check Zig Formatting run: | find geode/src -name "*.zig" | xargs zig fmt --check - name: Lint Go Client working-directory: geode-client-go run: | go fmt ./... go vet ./... golangci-lint run - name: Lint Python Client working-directory: geode-client-python run: | black --check . pylint geode_client/ mypy geode_client/ - name: Lint Rust Client working-directory: geode-client-rust run: | cargo fmt -- --check cargo clippy -- -D warnings </code></pre></div>Static Analysis: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"> - name: Run Static Analysis run: | # Memory safety checks zig build -Dcheck # Security scanning semgrep --config=auto geode/src/ # Dependency audit cd geode-client-go && go list -m all | nancy sleuth cd geode-client-rust && cargo audit </code></pre></div>CANARY Marker Validation: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"> - name: Validate CANARY Markers run: | # Ensure all implementations have CANARY markers python scripts/validate-canary-markers.py \ --dir geode/src \ --require-evidence </code></pre></div> <h3 id="advanced-ci-patterns" class="position-relative d-flex align-items-center group"> Advanced CI Patterns <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="advanced-ci-patterns" aria-haspopup="dialog" aria-label="Share link: Advanced CI Patterns"> Share link </button> </h3> <h4 id="parallel-test-execution" class="position-relative d-flex align-items-center group"> Parallel Test Execution <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="parallel-test-execution" aria-haspopup="dialog" aria-label="Share link: Parallel Test Execution"> Share link </button> </h4>Geode parallelizes tests for faster feedback: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">test-parallel: name: Parallel Tests runs-on: ubuntu-24.04 strategy: fail-fast: false matrix: shard: [1, 2, 3, 4, 5, 6, 7, 8] steps: - name: Run Test Shard ${{ matrix.shard }} run: | # Split tests into 8 shards zig build test -Dshard=${{ matrix.shard }} -Dtotal-shards=8 </code></pre></div>Benefits: <ul> <li>8-minute test suite runs in 1-2 minutes</li> <li>Early failure detection</li> <li>Better resource utilization</li> </ul> <h4 id="incremental-testing" class="position-relative d-flex align-items-center group"> Incremental Testing <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="incremental-testing" aria-haspopup="dialog" aria-label="Share link: Incremental Testing"> Share link </button> </h4>Only test affected components: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"> - name: Detect Changed Files id: changed run: | # Get changed files since last commit CHANGED=$(git diff --name-only HEAD~1 HEAD) echo "files=$CHANGED" >> $GITHUB_OUTPUT - name: Run Targeted Tests run: | if echo "${{ steps.changed.outputs.files }}" | grep -q "geode/src/parser"; then # Parser changed - run parser tests zig build test -Dtest-filter="parser.*" fi if echo "${{ steps.changed.outputs.files }}" | grep -q "geode-client-python"; then # Python client changed cd geode-client-python && pytest fi </code></pre></div> <h4 id="dependency-caching" class="position-relative d-flex align-items-center group"> Dependency Caching <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="dependency-caching" aria-haspopup="dialog" aria-label="Share link: Dependency Caching"> Share link </button> </h4>Cache dependencies to speed up builds: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"> - name: Cache Zig Dependencies uses: actions/cache@v4 with: path: | ~/.cache/zig geode/zig-cache geode/zig-out key: ${{ runner.os }}-zig-${{ hashFiles('**/build.zig.zon') }} restore-keys: | ${{ runner.os }}-zig- - name: Cache Go Modules uses: actions/cache@v4 with: path: ~/go/pkg/mod key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }} - name: Cache Rust Dependencies uses: actions/cache@v4 with: path: | ~/.cargo/registry ~/.cargo/git geode-client-rust/target key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }} </code></pre></div> <h4 id="test-result-reporting" class="position-relative d-flex align-items-center group"> Test Result Reporting <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="test-result-reporting" aria-haspopup="dialog" aria-label="Share link: Test Result Reporting"> Share link </button> </h4>Generate comprehensive test reports: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"> - name: Generate Test Report if: always() run: | python scripts/generate-test-report.py \ --input test-results/ \ --output test-report.html \ --format html - name: Publish Test Results uses: EnricoMi/publish-unit-test-result-action@v2 if: always() with: files: | geode/test-results/**/*.xml geode-client-*/test-results/**/*.xml geode-test-harness/reports/**/*.xml - name: Comment PR with Results if: github.event_name == 'pull_request' uses: actions/github-script@v7 with: script: | const fs = require('fs'); const report = fs.readFileSync('test-report.html', 'utf8'); github.rest.issues.createComment({ issue_number: context.issue.number, owner: context.repo.owner, repo: context.repo.repo, body: report }); </code></pre></div> <h3 id="ci-best-practices-for-geode" class="position-relative d-flex align-items-center group"> CI Best Practices for Geode <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="ci-best-practices-for-geode" aria-haspopup="dialog" aria-label="Share link: CI Best Practices for Geode"> Share link </button> </h3> <h4 id="1-fail-fast" class="position-relative d-flex align-items-center group"> 1. Fail Fast <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="1-fail-fast" aria-haspopup="dialog" aria-label="Share link: 1. Fail Fast"> Share link </button> </h4>Prioritize quick checks first: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">jobs: # Run fast checks first quick-checks: runs-on: ubuntu-24.04 steps: - name: Lint (30 seconds) - name: Format Check (20 seconds) - name: Compile Check (1 minute) # Run expensive tests only if quick checks pass comprehensive-tests: needs: quick-checks steps: - name: Full Test Suite (8 minutes) - name: Performance Tests (15 minutes) </code></pre></div> <h4 id="2-keep-ci-fast" class="position-relative d-flex align-items-center group"> 2. Keep CI Fast <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="2-keep-ci-fast" aria-haspopup="dialog" aria-label="Share link: 2. Keep CI Fast"> Share link </button> </h4>Target: Feedback within 10 minutes Optimization strategies: <ul> <li>Cache aggressively (builds, dependencies, test data)</li> <li>Parallelize independent tests</li> <li>Use incremental testing</li> <li>Run expensive tests nightly instead of per-commit</li> </ul> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">on: push: branches: [main] pull_request: # Fast tests on PR types: [opened, synchronize] schedule: # Comprehensive tests nightly - cron: '0 2 * * *' </code></pre></div> <h4 id="3-make-ci-reproducible" class="position-relative d-flex align-items-center group"> 3. Make CI Reproducible <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="3-make-ci-reproducible" aria-haspopup="dialog" aria-label="Share link: 3. Make CI Reproducible"> Share link </button> </h4>Pin all versions: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">env: ZIG_VERSION: 0.1.0 GO_VERSION: 1.24.0 PYTHON_VERSION: 3.11.8 RUST_VERSION: 1.70.0 # Pin action versions CHECKOUT_VERSION: v4 SETUP_GO_VERSION: v5 </code></pre></div>Use deterministic builds: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"># Zig builds are deterministic by default zig build -Doptimize=ReleaseSafe # Go builds with reproducibility go build -trimpath -ldflags="-buildid=" </code></pre></div> <h4 id="4-test-in-clean-environments" class="position-relative d-flex align-items-center group"> 4. Test in Clean Environments <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="4-test-in-clean-environments" aria-haspopup="dialog" aria-label="Share link: 4. Test in Clean Environments"> Share link </button> </h4>Avoid environment pollution: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"> - name: Run Tests in Clean Environment run: | # Each test gets fresh database for test in tests/*; do # Clean state rm -rf /tmp/geode-test-data mkdir -p /tmp/geode-test-data # Run isolated GEODE_DATA_DIR=/tmp/geode-test-data $test done </code></pre></div> <h4 id="5-monitor-ci-health" class="position-relative d-flex align-items-center group"> 5. Monitor CI Health <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="5-monitor-ci-health" aria-haspopup="dialog" aria-label="Share link: 5. Monitor CI Health"> Share link </button> </h4>Track CI metrics: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"># scripts/ci-metrics.py from datetime import datetime from prometheus_client import Counter, Histogram, push_to_gateway ci_duration = Histogram('ci_duration_seconds', 'CI duration', ['job']) ci_failures = Counter('ci_failures_total', 'CI failures', ['job']) def report_ci_metrics(job_name, duration, success): ci_duration.labels(job=job_name).observe(duration) if not success: ci_failures.labels(job=job_name).inc() push_to_gateway('monitoring.geodedb.com:9091', job='ci-metrics', registry=registry) </code></pre></div> <h3 id="debugging-ci-failures" class="position-relative d-flex align-items-center group"> Debugging CI Failures <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="debugging-ci-failures" aria-haspopup="dialog" aria-label="Share link: Debugging CI Failures"> Share link </button> </h3> <h4 id="access-build-logs" class="position-relative d-flex align-items-center group"> Access Build Logs <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="access-build-logs" aria-haspopup="dialog" aria-label="Share link: Access Build Logs"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"># Download logs from GitHub Actions gh run view <run-id> --log # Or via API curl -L \ -H "Accept: application/vnd.github+json" \ -H "Authorization: Bearer $GITHUB_TOKEN" \ https://api.github.com/repos/devnw/geode/actions/runs/<run-id>/logs </code></pre></div> <h4 id="reproduce-locally" class="position-relative d-flex align-items-center group"> Reproduce Locally <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="reproduce-locally" aria-haspopup="dialog" aria-label="Share link: Reproduce Locally"> Share link </button> </h4>Use act to run GitHub Actions locally: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"># Install act brew install act # macOS # or curl https://raw.githubusercontent.com/nektos/act/master/install.sh | bash # Run specific job act -j unit-tests # Run entire workflow act push </code></pre></div> <h4 id="debug-mode" class="position-relative d-flex align-items-center group"> Debug Mode <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="debug-mode" aria-haspopup="dialog" aria-label="Share link: Debug Mode"> Share link </button> </h4>Enable debug logging: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"> - name: Enable Debug Logging if: runner.debug run: | export GEODE_LOG_LEVEL=debug export GEODE_TRACE=1 zig build test -Dlog-level=debug </code></pre></div>Trigger with: <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash">gh workflow run ci.yml -f debug=true </code></pre></div> <h4 id="common-ci-issues" class="position-relative d-flex align-items-center group"> Common CI Issues <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="common-ci-issues" aria-haspopup="dialog" aria-label="Share link: Common CI Issues"> Share link </button> </h4>Issue 1: Flaky Tests <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"># Mark flaky tests with retries @pytest.mark.flaky(reruns=3, reruns_delay=1) async def test_concurrent_transactions(): # Test that occasionally fails due to timing pass </code></pre></div>Issue 2: Resource Exhaustion <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"> - name: Clean Up Resources if: always() run: | # Stop all Geode processes pkill -9 geode || true # Clean temp data rm -rf /tmp/geode-* # Free memory sync; echo 3 > /proc/sys/vm/drop_caches || true </code></pre></div>Issue 3: Timeout Issues <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml">jobs: test: timeout-minutes: 30 # Prevent hanging builds steps: - name: Run Tests timeout-minutes: 20 # Per-step timeout run: make test </code></pre></div> <h3 id="ci-security-considerations" class="position-relative d-flex align-items-center group"> CI Security Considerations <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="ci-security-considerations" aria-haspopup="dialog" aria-label="Share link: CI Security Considerations"> Share link </button> </h3> <h4 id="secret-management" class="position-relative d-flex align-items-center group"> Secret Management <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="secret-management" aria-haspopup="dialog" aria-label="Share link: Secret Management"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"> - name: Configure Secrets env: GEODE_LICENSE_KEY: ${{ secrets.GEODE_LICENSE_KEY }} AWS_SECRET_KEY: ${{ secrets.AWS_SECRET_KEY }} run: | # Never echo secrets # Never log secrets # Use secrets only where needed </code></pre></div> <h4 id="dependency-scanning" class="position-relative d-flex align-items-center group"> Dependency Scanning <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="dependency-scanning" aria-haspopup="dialog" aria-label="Share link: Dependency Scanning"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"> - name: Scan Dependencies run: | # Scan for vulnerabilities cd geode-client-go && govulncheck ./... cd geode-client-rust && cargo audit cd geode-client-python && safety check </code></pre></div> <h4 id="code-scanning" class="position-relative d-flex align-items-center group"> Code Scanning <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="code-scanning" aria-haspopup="dialog" aria-label="Share link: Code Scanning"> Share link </button> </h4><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"> - name: CodeQL Analysis uses: github/codeql-action/analyze@v3 with: languages: go, python, cpp queries: security-and-quality </code></pre></div> <h3 id="related-topics" class="position-relative d-flex align-items-center group"> Related Topics <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="related-topics" aria-haspopup="dialog" aria-label="Share link: Related Topics"> Share link </button> </h3><ul> <li><a href="/tags/ci-cd/" >CI/CD</a> : Complete CI/CD pipeline</li> <li><a href="/tags/unit-tests/" >Unit Tests</a> : Unit testing strategies</li> <li><a href="/tags/integration-tests/" >Integration Tests</a> : Integration testing</li> <li><a href="/tags/deployment/" >Deployment</a> : Deployment automation</li> <li><a href="/tags/containers/" >Containers</a> : Container-based CI</li> <li><a href="/tags/cloud/" >Cloud</a> : Cloud-based CI/CD platforms</li> </ul> <h3 id="further-reading" class="position-relative d-flex align-items-center group"> Further Reading <button type="button" class="h-share btn btn-link p-0 text-decoration-none link-secondary opacity-50 hover-opacity-100 transition-all ms-1" data-share-target="further-reading" aria-haspopup="dialog" aria-label="Share link: Further Reading"> Share link </button> </h3><ul> <li>CI Best Practices: <code>/docs/development/ci-best-practices/</code></li> <li>Test Automation Guide: <code>/docs/development/test-automation/</code></li> <li>GitHub Actions Reference: <code>/docs/operations/github-actions/</code></li> <li>Build Optimization: <code>/docs/development/build-optimization/</code></li> <li>Quality Assurance: <code>/docs/development/quality-assurance/</code></li> </ul>

Popular

Related Articles