Testing Guide

Geode maintains comprehensive test coverage with 95.9% extended integration test pass rate (3395/3540 tests) and 100% GQL compliance (see conformance profile). This guide covers all testing frameworks, strategies, and best practices.

Overview

Test Coverage Summary

Current Status (as of February 2026):

Extended Integration Tests (full suite):
  Total: 3,540 tests
  Passing: 3,395 tests (95.9%)
  Failing: 110 tests (3.1%)
  Skipped: 35 tests (1.0%)

Integration Tests (geodetestlab core):
  Total: 1,688 tests
  Passing: 1,644 tests (97.4%)

Unit Tests (Zig):
  Total: 393 test blocks
  Passing: 393 tests (100%)

GQL Conformance Profile:
  Scope: ISO/IEC 39075:2024 compliance (see conformance profile)
  Status: tracked in CI with published diagnostics
  Note: spec timeout increased to 30s for CI reliability (QUIC handshake latency)

CANARY Governance:
  Total: 1,735 markers
  Requirements: 2,190+
  Tested: 81.4%
  Benched: 6.0%
  Implemented: 5.7%
  Exempt: 7.7%

Coverage by Category:

Category	Tests	Pass Rate
Authentication & Authorization	285	100%
User Management	134	100%
Role & Policy Management	178	100%
Query Execution	912	97.3%
Server Operations	468	93.8%
CLI Commands	356	98.6%
GQL Conformance (profile)	see profile	see profile
Storage & Backup	246	97.2%
Distributed Systems	174	91.4%
Security Features	312	100%
Advanced Features	166	93.4%
Rapid Sequential Writes (#479)	29	100%
Regression (self-ref pointer, write-all I/O, DecimalValue)	280	94.6%

Testing Frameworks

Geode uses multiple complementary testing frameworks:

Zig Unit Tests (zig test) - Fast, inline tests for individual functions
geodetestlab - Python-based integration testing with YAML specifications
Regression Tests - Shell scripts for specific bug scenarios
Fuzz Testing - Randomized input testing for parser and storage
Benchmark Tests - Performance regression detection
GQL Conformance Tests - Profile-based GQL conformance

Quick Start

Running All Tests

# From repository root
cd /home/benji/src/gl/devnw/codepros/geode/geode

# Quick verification (unit tests only)
make test

# Integration tests
make integration-test

# Full geodetestlab suite
python3 scripts/test/extended_test_runner.py

# Comprehensive test suite (all tests)
make geodetestlab-comprehensive

# Specific test spec
python3 scripts/test/extended_test_runner.py --specs geodetestlab/specs/geode_gql_compliance.yml

# Verbose output
python3 scripts/test/extended_test_runner.py --verbose

# Generate test report
python3 scripts/test/extended_test_runner.py --report test-results.json

Running Specific Test Categories

# Authentication tests only
python3 scripts/test/extended_test_runner.py --specs geodetestlab/specs/auth*.yml

# GQL compliance tests
python3 scripts/test/extended_test_runner.py --specs geodetestlab/specs/geode_gql_compliance.yml

# Regression tests
./tests/comprehensive_regression_test.sh
./tests/limit_skip_regression_test.sh
./tests/property_ordering_regression_test.sh
./tests/count_aggregation_regression_test.sh

# Fuzz tests
make fuzz

# Benchmark tests
make bench

Test Environment Setup

Prerequisites:

# Python 3.9+ with dependencies
python3 --version  # Should be 3.9 or higher
pip install pyyaml  # For geodetestlab

# Zig 0.1.0+ for unit tests
zig version  # Should be 0.1.0 or higher

# Build Geode binary
make build

# Verify build
./zig-out/bin/geode --version

Environment Variables:

# Enable debug output
export GEODE_DEBUG=1

# Capture server stderr in tests
export GEODE_CAPTURE_SERVER_STDERR=1

# Set test timeout (seconds)
export GEODE_TEST_TIMEOUT=30

# Enable JSON logging
export GEODE_LOG_JSON=1

Zig Unit Tests

Overview

Unit tests are embedded directly in Zig source files using test blocks. They provide fast, fine-grained testing of individual functions and modules.

Writing Unit Tests

Basic Test Structure:

const std = @import("std");
const testing = std.testing;

pub fn add(a: i32, b: i32) i32 {
    return a + b;
}

test "add: basic addition" {
    try testing.expectEqual(@as(i32, 5), add(2, 3));
    try testing.expectEqual(@as(i32, 0), add(-1, 1));
    try testing.expectEqual(@as(i32, -5), add(-2, -3));
}

Testing with Allocators:

test "ArrayList: append and get" {
    const allocator = testing.allocator;
    var list = std.ArrayList(i32).init(allocator);
    defer list.deinit();

    try list.append(42);
    try list.append(100);

    try testing.expectEqual(@as(i32, 42), list.items[0]);
    try testing.expectEqual(@as(i32, 100), list.items[1]);
    try testing.expectEqual(@as(usize, 2), list.items.len);
}

Testing Error Conditions:

pub fn divide(a: i32, b: i32) !i32 {
    if (b == 0) return error.DivisionByZero;
    return @divTrunc(a, b);
}

test "divide: division by zero" {
    try testing.expectError(error.DivisionByZero, divide(10, 0));
}

test "divide: successful division" {
    try testing.expectEqual(@as(i32, 5), try divide(10, 2));
    try testing.expectEqual(@as(i32, -3), try divide(-9, 3));
}

Testing Complex Structures:

const Node = struct {
    id: u64,
    name: []const u8,
    properties: std.StringHashMap([]const u8),

    pub fn create(allocator: std.mem.Allocator, id: u64, name: []const u8) !Node {
        var props = std.StringHashMap([]const u8).init(allocator);
        return Node{
            .id = id,
            .name = name,
            .properties = props,
        };
    }

    pub fn deinit(self: *Node) void {
        self.properties.deinit();
    }
};

test "Node: creation and properties" {
    const allocator = testing.allocator;
    var node = try Node.create(allocator, 1, "test_node");
    defer node.deinit();

    try testing.expectEqual(@as(u64, 1), node.id);
    try testing.expectEqualStrings("test_node", node.name);

    try node.properties.put("key1", "value1");
    try testing.expectEqualStrings("value1", node.properties.get("key1").?);
}

Running Unit Tests

# Run all tests in a file
zig test src/storage/heap.zig

# Run specific test by name filter
zig test src/storage/heap.zig --test-filter "allocate"

# Run with verbose output
zig test src/storage/heap.zig --summary all

# Run with debug symbols
zig test src/storage/heap.zig -O Debug

# Run all project tests
make test

# Run with coverage (using kcov or similar)
zig test src/storage/heap.zig --test-coverage

Test Organization

Best Practices:

Colocate tests - Keep tests near the code they test
Descriptive names - Use format “component: specific behavior”
Test edge cases - Include boundary conditions and error paths
Clean up resources - Always use defer for cleanup
Avoid dependencies - Tests should be independent and isolated

Example Test Organization:

// src/storage/heap.zig

pub const Heap = struct {
    // ... implementation ...
};

// Unit tests for Heap
test "Heap: initialization" { ... }
test "Heap: allocate page" { ... }
test "Heap: deallocate page" { ... }
test "Heap: grow heap" { ... }
test "Heap: out of memory" { ... }
test "Heap: fragmentation handling" { ... }

Common Patterns

Testing Allocations:

test "MyStruct: no memory leaks" {
    const allocator = testing.allocator;

    var obj = try MyStruct.init(allocator);
    defer obj.deinit(); // Cleanup automatically checked

    // Use obj...

    // testing.allocator will fail if memory leaked
}

Testing Concurrency:

test "ThreadPool: concurrent task execution" {
    const allocator = testing.allocator;
    var pool = try ThreadPool.init(allocator, 4);
    defer pool.deinit();

    var counter = std.atomic.Atomic(u32).init(0);

    var i: usize = 0;
    while (i < 100) : (i += 1) {
        try pool.spawn(struct {
            fn increment(c: *std.atomic.Atomic(u32)) void {
                _ = c.fetchAdd(1, .SeqCst);
            }
        }.increment, .{&counter});
    }

    pool.waitAll();
    try testing.expectEqual(@as(u32, 100), counter.load(.SeqCst));
}

Parameterized Tests:

test "hash_function: various inputs" {
    const test_cases = [_]struct {
        input: []const u8,
        expected: u64,
    }{
        .{ .input = "hello", .expected = 0x12345678 },
        .{ .input = "world", .expected = 0x87654321 },
        .{ .input = "", .expected = 0 },
    };

    for (test_cases) |tc| {
        const result = hash_function(tc.input);
        try testing.expectEqual(tc.expected, result);
    }
}

geodetestlab Framework

Overview

geodetestlab is Geode’s primary integration testing framework. It uses YAML specification files to define test cases that execute against a running Geode server.

Key Features:

YAML-based test specifications
Variable substitution (server address, secrets, etc.)
JSON output validation
Regex pattern matching
Exit code verification
Comprehensive error reporting

Test Specification Format

Basic Structure:

name: "test-suite-name"
description: "Description of test suite"
tests:
  - name: "test-case-name"
    description: "What this test validates"
    args: ["command", "subcommand", "--flag", "value"]
    expect:
      exit_code: 0
      stdout_contains: ["expected output"]
      stderr_contains: []
      stdout_regex: ["pattern.*to.*match"]
      stdout_json_assert:
        - path: "result.status"
          equals: "success"

Complete Example:

name: "basic-query-tests"
description: "Basic query execution tests"
tests:
  - name: "simple-return"
    description: "Execute simple RETURN query"
    args: ["query", "RETURN 1 AS x", "--server", "${server}"]
    expect:
      exit_code: 0
      stdout_contains: ["x"]
      stdout_json_assert:
        - path: "result.columns[0].name"
          equals: "x"
        - path: "result.rows[0].x"
          equals: 1

  - name: "query-with-where"
    description: "Query with WHERE clause"
    args: ["query", "MATCH (n:Person) WHERE n.age > 30 RETURN n.name", "--server", "${server}"]
    expect:
      exit_code: 0
      stdout_regex: [".*Person.*"]

  - name: "invalid-syntax"
    description: "Invalid query should fail"
    args: ["query", "INVALID SYNTAX", "--server", "${server}"]
    expect:
      exit_code: 1
      stderr_contains: ["syntax error", "parse error"]

Variable Substitution

geodetestlab supports dynamic variable substitution:

Built-in Variables:

tests:
  - name: "server-connection"
    args: ["query", "RETURN 1", "--server", "${server}"]
    # ${server} → 127.0.0.1:3141

  - name: "server-host-only"
    args: ["--host", "${server:host}"]
    # ${server:host} → 127.0.0.1

  - name: "server-port-only"
    args: ["--port", "${server:port}"]
    # ${server:port} → 3141

  - name: "authenticated-query"
    args: ["query", "RETURN 1", "--server", "${server}", "--token", "${secret:api_token}"]
    # ${secret:api_token} → generated test token

  - name: "user-password"
    args: ["user", "create", "--password", "${secret:user_password}"]
    # ${secret:user_password} → generated test password

Custom Variables (via environment):

# Set custom variables
export GEODE_TEST_VAR_GRAPH_NAME="test_graph"

# Use in test spec
args: ["use", "graph", "${custom:GRAPH_NAME}"]

JSON Assertions

Path-based Assertions:

stdout_json_assert:
  # Exact equality
  - path: "result.status"
    equals: "success"

  # Type checking
  - path: "result.count"
    type: "integer"

  # Existence checking
  - path: "result.data"
    exists: true

  # Array length
  - path: "result.rows"
    length: 5

  # Nested paths
  - path: "result.rows[0].user.name"
    equals: "Alice"

  # Contains substring
  - path: "result.message"
    contains: "completed successfully"

  # Regex match
  - path: "result.id"
    matches: "^[0-9a-f]{8}-[0-9a-f]{4}-"

Complex Assertions:

stdout_json_assert:
  # Validate schema
  - path: "result.columns"
    schema:
      type: array
      items:
        type: object
        required: [name, type]
        properties:
          name: {type: string}
          type: {type: string}

  # Multiple conditions (AND)
  - path: "result.count"
    all:
      - greater_than: 0
      - less_than: 100

  # Any condition (OR)
  - path: "result.status"
    any:
      - equals: "success"
      - equals: "completed"

Running geodetestlab Tests

Basic Usage:

# Run all tests
python3 scripts/test/extended_test_runner.py

# Run specific spec file
python3 scripts/test/extended_test_runner.py --specs geodetestlab/specs/auth_tests.yml

# Run multiple spec files
python3 scripts/test/extended_test_runner.py --specs \
  geodetestlab/specs/auth_tests.yml \
  geodetestlab/specs/query_tests.yml

# Verbose output
python3 scripts/test/extended_test_runner.py --verbose

# Generate JSON report
python3 scripts/test/extended_test_runner.py --report test-results.json

# Generate HTML report
python3 scripts/test/extended_test_runner.py --html-report test-results.html

# Run with specific server address
python3 scripts/test/extended_test_runner.py --server 127.0.0.1:8443

# Stop on first failure
python3 scripts/test/extended_test_runner.py --fail-fast

Advanced Options:

# Run with timeout (seconds)
python3 scripts/test/extended_test_runner.py --timeout 60

# Run specific test by name pattern
python3 scripts/test/extended_test_runner.py --filter "auth.*login"

# Exclude specific tests
python3 scripts/test/extended_test_runner.py --exclude "slow_tests"

# Parallel execution (4 workers)
python3 scripts/test/extended_test_runner.py --parallel 4

# Retry failed tests
python3 scripts/test/extended_test_runner.py --retry 3

Test Organization

Spec File Naming Convention:

geodetestlab/specs/
├── auth_basic.yml           # Basic authentication
├── auth_mfa.yml             # Multi-factor authentication
├── query_basic.yml          # Basic queries
├── query_aggregation.yml    # Aggregation queries
├── crud_nodes.yml           # Node CRUD operations
├── crud_relationships.yml   # Relationship CRUD
├── txn_isolation.yml        # Transaction isolation
├── backup_restore.yml       # Backup/restore operations
├── distributed_*.yml        # Distributed features
├── rapid_sequential_writes.yml # Bulk write regression (#479)
└── geode_gql_compliance.yml # ISO GQL compliance

Test Categories:

Authentication (auth_*.yml) - Login, MFA, tokens
Authorization (authz_*.yml) - Permissions, RBAC, ABAC
Query Execution (query_*.yml) - MATCH, RETURN, WHERE, etc.
CRUD Operations (crud_*.yml) - CREATE, DELETE, UPDATE
Transactions (txn_*.yml) - BEGIN, COMMIT, ROLLBACK
Storage (storage_*.yml) - Persistence, indexes
Backup (backup_*.yml) - Backup and restore
Security (security_*.yml) - TDE, RLS, audit
Distributed (distributed_*.yml) - Federation, sharding
Compliance (*_compliance.yml) - Standards conformance
Rapid Sequential Writes (rapid_sequential_writes.yml) - Bulk MERGE/CREATE, relationship chains, mixed operations, data integrity, and stress tests for the gRPC aarch64 partial write segfault fix (#479)

Writing New Tests

Step 1: Create Spec File

# Create new spec file
touch geodetestlab/specs/my_feature_tests.yml

Step 2: Define Test Suite

name: "my-feature-tests"
description: "Tests for my new feature"
tests:
  - name: "feature-basic-usage"
    description: "Basic usage of the feature"
    args: ["my-command", "--option", "value"]
    expect:
      exit_code: 0
      stdout_contains: ["success"]

Step 3: Run Tests

# Test your new spec
python3 scripts/test/extended_test_runner.py --specs geodetestlab/specs/my_feature_tests.yml --verbose

Step 4: Validate Output

# Check test results
echo $?  # Should be 0 for success

# Review detailed output
cat test-results.json

Debugging Failed Tests

Enable Verbose Mode:

python3 scripts/test/extended_test_runner.py --verbose --specs geodetestlab/specs/failing_test.yml

Capture Server Output:

export GEODE_CAPTURE_SERVER_STDERR=1
python3 scripts/test/extended_test_runner.py --specs geodetestlab/specs/failing_test.yml

Run Single Test:

# Temporarily comment out other tests in spec file
tests:
  # - name: "test1" ...
  # - name: "test2" ...
  - name: "failing-test"
    # ... only this test will run

Manual Execution:

# Run the exact command from test spec manually
./zig-out/bin/geode query "MATCH (n) RETURN n" --server 127.0.0.1:3141 --verbose

Integration with CI/CD

GitLab CI Configuration (.gitlab-ci.yml):

test-geodetestlab:
  stage: test
  image: registry.gitlab.com/devnw/ci-catalog/zig:0.1.0
  before_script:
    - apt-get update && apt-get install -y python3 python3-pip
    - pip3 install pyyaml
    - make build
  script:
    - python3 scripts/test/extended_test_runner.py --report test-results.json
  artifacts:
    reports:
      junit: test-results.xml
    paths:
      - test-results.json
      - test-results.html
    when: always
    expire_in: 30 days
  coverage: '/Total coverage: (\d+\.\d+)%/'

Regression Tests

Overview

Regression tests are shell scripts that validate specific bug fixes and ensure they don’t reoccur. Each regression test focuses on a particular scenario.

Available Regression Suites

comprehensive_regression_test.sh (34 scenarios):

./tests/comprehensive_regression_test.sh

# Categories tested:
# - Basic queries and filters
# - Path patterns and relationships
# - Aggregations and grouping
# - Order by and pagination
# - Property access and null handling
# - Set operations (UNION, INTERSECT)
# - Transaction management
# - Error handling

limit_skip_regression_test.sh (23 scenarios):

./tests/limit_skip_regression_test.sh

# Categories tested:
# - LIMIT clause variations
# - OFFSET/SKIP clause variations
# - Combined LIMIT and OFFSET
# - ORDER BY with pagination
# - Edge cases (zero, negative, large values)

property_ordering_regression_test.sh (10 scenarios):

./tests/property_ordering_regression_test.sh

# Categories tested:
# - Property insertion order preservation
# - JSON output property ordering
# - Map literal property ordering
# - UPDATE property ordering
# - Multi-property consistency

count_aggregation_regression_test.sh (8 scenarios):

./tests/count_aggregation_regression_test.sh

# Categories tested:
# - COUNT(*) aggregation
# - COUNT(DISTINCT) aggregation
# - COUNT with GROUP BY
# - COUNT with WHERE filtering
# - COUNT with null values

Writing Regression Tests

Template:

#!/bin/bash
set -e

# Test configuration
TEST_NAME="my_regression_test"
SERVER="127.0.0.1:3141"
GEODE_BIN="./zig-out/bin/geode"

# Setup
echo "Setting up test environment..."
$GEODE_BIN serve --listen $SERVER &
SERVER_PID=$!
sleep 2

# Cleanup on exit
trap "kill $SERVER_PID 2>/dev/null || true" EXIT

# Test case 1
echo "Test 1: Description of what we're testing"
result=$($GEODE_BIN query "MATCH (n) RETURN n" --server $SERVER)
if [[ "$result" != *"expected output"* ]]; then
    echo "FAIL: Test 1 failed"
    exit 1
fi
echo "PASS: Test 1"

# Test case 2
echo "Test 2: Another test case"
result=$($GEODE_BIN query "CREATE (n:Test)" --server $SERVER)
if [ $? -ne 0 ]; then
    echo "FAIL: Test 2 failed"
    exit 1
fi
echo "PASS: Test 2"

# Summary
echo "============================="
echo "All regression tests passed!"
echo "============================="
exit 0

Best Practices:

Self-contained - Each test starts and stops its own server
Cleanup - Use trap to ensure cleanup on exit
Clear output - Echo test names and results
Exit codes - Return 0 for success, non-zero for failure
Documentation - Comment what bug/issue the test validates

Running Regression Tests

# Run all regression suites
./tests/comprehensive_regression_test.sh
./tests/limit_skip_regression_test.sh
./tests/property_ordering_regression_test.sh
./tests/count_aggregation_regression_test.sh

# Run specific test with verbose output
bash -x ./tests/comprehensive_regression_test.sh

# Check exit code
./tests/comprehensive_regression_test.sh
echo $?  # Should be 0

# Run in CI
make regression-tests

Fuzz Testing

Overview

Fuzz testing uses randomized inputs to discover edge cases and potential crashes. Geode includes fuzz tests for critical components.

Fuzz Targets

Parser Fuzzing (fuzz/parser_fuzz.zig):

// Generates random GQL queries to test parser robustness
pub fn fuzz_parser(input: []const u8) !void {
    const allocator = std.heap.page_allocator;
    var parser = Parser.init(allocator, input);
    _ = parser.parse() catch |err| {
        // Expected - invalid input should fail gracefully
        return;
    };
}

Storage Fuzzing (fuzz/storage_fuzz.zig):

// Generates random storage operations
pub fn fuzz_storage(input: []const u8) !void {
    // Random node/relationship creation
    // Random property updates
    // Random deletions
    // Verify consistency after each operation
}

Execution Fuzzing (fuzz/execution_fuzz.zig):

// Generates random query execution scenarios
pub fn fuzz_execution(input: []const u8) !void {
    // Random MATCH patterns
    // Random WHERE conditions
    // Random aggregations
    // Verify results are valid
}

Running Fuzz Tests

# Run all fuzz tests
make fuzz

# Run specific fuzz target
zig build fuzz-parser

# Run with custom seed (reproducibility)
zig build fuzz-parser -Dseed=12345

# Run with time limit (10 minutes)
timeout 600 zig build fuzz-parser

# Run with AFL (American Fuzzy Lop)
afl-fuzz -i fuzz/inputs -o fuzz/outputs -- ./zig-out/bin/geode-fuzz-parser @@

Fuzz Test Configuration

// fuzz/config.zig
pub const FuzzConfig = struct {
    /// Maximum input size in bytes
    max_input_size: usize = 4096,

    /// Iteration count
    iterations: u64 = 10_000,

    /// Seed for PRNG (0 = random)
    seed: u64 = 0,

    /// Timeout per iteration (ms)
    timeout_ms: u64 = 100,

    /// Crash on first error
    fail_fast: bool = false,
};

Analyzing Fuzz Crashes

# Reproduce crash with specific input
zig build fuzz-parser -Dinput=crash_input.txt

# Debug crash
zig build fuzz-parser -O Debug -Dinput=crash_input.txt
gdb ./zig-out/bin/geode-fuzz-parser

# Minimize crash input
afl-tmin -i crash_input.txt -o minimized.txt -- ./zig-out/bin/geode-fuzz-parser @@

Benchmark Tests

Overview

Benchmark tests measure performance and detect regressions. Geode includes 150+ benchmark variants.

Benchmark Categories

Query Benchmarks:

// benchmark/query_bench.zig
pub fn benchmarkSimpleQuery() !void {
    const timer = try Timer.start();

    var i: usize = 0;
    while (i < 1000) : (i += 1) {
        _ = try executeQuery("MATCH (n) RETURN n LIMIT 1");
    }

    const elapsed = timer.read();
    std.debug.print("Simple query: {} ns/op\n", .{elapsed / 1000});
}

Storage Benchmarks:

pub fn benchmarkNodeInsertion() !void {
    const timer = try Timer.start();

    var i: usize = 0;
    while (i < 10000) : (i += 1) {
        _ = try storage.createNode(.{
            .id = i,
            .labels = &[_][]const u8{"Person"},
            .properties = &[_]Property{},
        });
    }

    const elapsed = timer.read();
    std.debug.print("Node insertion: {} ns/op\n", .{elapsed / 10000});
}

Index Benchmarks:

pub fn benchmarkIndexLookup() !void {
    // Setup: Create index with 100k entries
    var index = try BTreeIndex.init(allocator);
    var i: usize = 0;
    while (i < 100_000) : (i += 1) {
        try index.insert(i, i);
    }

    const timer = try Timer.start();

    // Benchmark: Lookup 10k random keys
    var prng = std.rand.DefaultPrng.init(12345);
    i = 0;
    while (i < 10_000) : (i += 1) {
        const key = prng.random().intRangeAtMost(usize, 0, 99_999);
        _ = try index.lookup(key);
    }

    const elapsed = timer.read();
    std.debug.print("Index lookup: {} ns/op\n", .{elapsed / 10_000});
}

Running Benchmarks

# Run all benchmarks
make bench

# Run specific benchmark
zig build bench-query

# Run with different optimization levels
zig build bench-query -O ReleaseFast
zig build bench-query -O ReleaseSafe

# Output to JSON
zig build bench-query --output benchmark-results.json

# Compare with baseline
./scripts/benchmark_compare.sh baseline.json current.json

Benchmark Output Format

{
  "benchmarks": [
    {
      "name": "simple_query",
      "iterations": 1000,
      "total_ns": 1250000,
      "ns_per_op": 1250,
      "ops_per_sec": 800000
    },
    {
      "name": "node_insertion",
      "iterations": 10000,
      "total_ns": 50000000,
      "ns_per_op": 5000,
      "ops_per_sec": 200000
    }
  ],
  "environment": {
    "os": "linux",
    "arch": "x86_64",
    "cpu": "Intel Core i7-9700K",
    "zig_version": "0.1.0"
  }
}

Performance Regression Detection

# Generate baseline
make bench > baseline-results.json

# After changes, compare
make bench > current-results.json
./scripts/detect_regressions.sh baseline-results.json current-results.json

# Example output:
# ✓ simple_query: 1250 ns/op (no change)
# ✗ node_insertion: 8000 ns/op (+60% regression)
# ✓ index_lookup: 450 ns/op (-10% improvement)

Test Coverage Analysis

Measuring Coverage

Zig Unit Test Coverage:

# Generate coverage data
zig test src/storage/heap.zig --test-coverage

# View coverage report (requires kcov or similar)
kcov coverage-output ./zig-test
firefox coverage-output/index.html

Integration Test Coverage:

# Run tests with coverage tracking
python3 scripts/test/extended_test_runner.py --coverage

# Generate coverage report
python3 scripts/test/generate_coverage_report.py

# Output:
# Total tests: 3540
# Passed: 3395 (95.9%)
# Failed: 110 (3.1%)
# Skipped: 35 (1.0%)
#
# Coverage by feature:
# - Authentication: 100% (285/285)
# - Query execution: 97.3% (888/912)
# - Storage: 97.2% (239/246)

Coverage Reports

HTML Report:

# Generate HTML coverage report
make coverage-html

# Open in browser
firefox coverage/index.html

Coverage Badges:

![Test Coverage](https://img.shields.io/badge/coverage-95.9%25-brightgreen)
![Unit Tests](https://img.shields.io/badge/unit%20tests-100%25-brightgreen)
![GQL Compliance](https://img.shields.io/badge/GQL%20compliance-100%25-blue)

Coverage Thresholds

CI/CD Enforcement:

# .gitlab-ci.yml
test-coverage:
  script:
    - make test-coverage
    - python3 scripts/check_coverage_threshold.py --min-coverage 95.0
  coverage: '/Total coverage: (\d+\.\d+)%/'

# scripts/check_coverage_threshold.py
import sys
import json

def check_coverage(min_threshold):
    with open('coverage-results.json') as f:
        data = json.load(f)

    coverage = data['pass_rate']
    print(f"Coverage: {coverage}%")

    if coverage < min_threshold:
        print(f"FAIL: Coverage {coverage}% below threshold {min_threshold}%")
        sys.exit(1)

    print(f"PASS: Coverage meets threshold")
    sys.exit(0)

if __name__ == '__main__':
    check_coverage(95.0)

CI/CD Integration

GitLab CI Pipeline

Complete Pipeline (.gitlab-ci.yml):

stages:
  - build
  - test
  - governance
  - deploy

variables:
  ZIG_VERSION: "0.1.0"
  GEODE_LOG_LEVEL: "info"

# Build stage
build:
  stage: build
  image: registry.gitlab.com/devnw/ci-catalog/zig:${ZIG_VERSION}
  script:
    - make build
    - ./zig-out/bin/geode --version
  artifacts:
    paths:
      - zig-out/bin/geode
    expire_in: 1 day

# Unit tests
test-unit:
  stage: test
  image: registry.gitlab.com/devnw/ci-catalog/zig:${ZIG_VERSION}
  script:
    - make test
  coverage: '/(\d+)\/(\d+) tests passed/'

# Integration tests
test-integration:
  stage: test
  needs: [build]
  image: registry.gitlab.com/devnw/ci-catalog/zig:${ZIG_VERSION}
  before_script:
    - apt-get update && apt-get install -y python3 python3-pip
    - pip3 install pyyaml
  script:
    - python3 scripts/test/extended_test_runner.py --report test-results.json
  artifacts:
    reports:
      junit: test-results.xml
    paths:
      - test-results.json
      - test-results.html
    when: always
    expire_in: 7 days
  coverage: '/Pass rate: (\d+\.\d+)%/'

# Regression tests
test-regression:
  stage: test
  needs: [build]
  script:
    - ./tests/comprehensive_regression_test.sh
    - ./tests/limit_skip_regression_test.sh
    - ./tests/property_ordering_regression_test.sh
    - ./tests/count_aggregation_regression_test.sh

# GQL compliance tests
test-gql-compliance:
  stage: test
  needs: [build]
  script:
    - python3 scripts/test/extended_test_runner.py --specs geodetestlab/specs/geode_gql_compliance.yml
  allow_failure: false  # Must pass

# Fuzz tests (scheduled)
test-fuzz:
  stage: test
  only:
    - schedules
  script:
    - timeout 600 make fuzz
  allow_failure: true

# Benchmark tests
test-benchmark:
  stage: test
  needs: [build]
  script:
    - make bench > benchmark-results.json
  artifacts:
    paths:
      - benchmark-results.json
    expire_in: 30 days

# Governance status check
governance-status:
  stage: governance
  script:
    - python3 tools/status_generate.py --repo . --out /tmp/status_check.csv
    - diff docs/status.csv /tmp/status_check.csv
  allow_failure: false

# Deploy (production)
deploy-production:
  stage: deploy
  only:
    - main
  when: manual
  environment:
    name: production
    url: https://geode.example.com
  script:
    - echo "Deploying to production..."
    - ./scripts/deploy.sh production

GitHub Actions (Alternative)

# .github/workflows/test.yml
name: Test Suite

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: goto-bus-stop/setup-zig@v2
        with:
          version: 0.1.0
      - name: Run unit tests
        run: make test

  integration-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: goto-bus-stop/setup-zig@v2
        with:
          version: 0.1.0
      - uses: actions/setup-python@v4
        with:
          python-version: '3.9'
      - name: Install dependencies
        run: pip install pyyaml
      - name: Build Geode
        run: make build
      - name: Run integration tests
        run: python3 scripts/test/extended_test_runner.py --report test-results.json
      - name: Upload test results
        uses: actions/upload-artifact@v3
        with:
          name: test-results
          path: test-results.*

  gql-compliance:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: goto-bus-stop/setup-zig@v2
      - name: Build Geode
        run: make build
      - name: Run GQL compliance tests
        run: python3 scripts/test/extended_test_runner.py --specs geodetestlab/specs/geode_gql_compliance.yml

Test Artifacts

Collecting Test Results:

artifacts:
  reports:
    junit: test-results.xml
  paths:
    - test-results.json
    - test-results.html
    - coverage-results/
    - benchmark-results.json
  when: always
  expire_in: 7 days

Test Report Format (JUnit XML):

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="Geode Tests" tests="3540" failures="110" errors="0" time="245.67">
  <testsuite name="auth_basic" tests="145" failures="0" errors="0" time="12.45">
    <testcase name="login_success" time="0.12"/>
    <testcase name="login_invalid_password" time="0.09"/>
    <!-- ... -->
  </testsuite>
  <testsuite name="query_execution" tests="456" failures="4" errors="0" time="67.89">
    <testcase name="simple_match" time="0.15"/>
    <testcase name="complex_where_clause" time="0.23">
      <failure message="Expected 5 results, got 4">
        Expected: 5
        Actual: 4
      </failure>
    </testcase>
    <!-- ... -->
  </testsuite>
</testsuites>

Test Data Management

Test Datasets

Loading Test Data:

# Load example datasets
make load-dataset

# Available datasets:
# - Social Network: 10,000+ users, posts, groups, events
# - IoT Network: 25,000+ devices, sensors, telemetry
# - Financial Network: 50,000 accounts, 250,000 transactions
# - Geographic Data: 100 cities, 25,000 POIs

# Load specific dataset
./scripts/load_dataset.sh social_network
./scripts/load_dataset.sh iot_network

# Run dataset integration tests
make dataset-integration-test

Test Data Fixtures:

# geodetestlab/fixtures/test_users.yml
users:
  - id: 1
    name: "Alice"
    email: "[email protected]"
    age: 30
  - id: 2
    name: "Bob"
    email: "[email protected]"
    age: 35

relationships:
  - type: "KNOWS"
    from: 1
    to: 2
    since: "2020-01-15"

Loading Fixtures in Tests:

tests:
  - name: "query-with-fixtures"
    setup:
      - load_fixture: "test_users.yml"
    args: ["query", "MATCH (u:User) WHERE u.age > 30 RETURN u.name"]
    expect:
      stdout_contains: ["Bob"]
    teardown:
      - cleanup_fixture: "test_users.yml"

Test Isolation

Database Cleanup:

# Reset database between test runs
geode admin reset-database --confirm

# Truncate specific graph
geode admin truncate-graph --graph test_graph

# Delete test data
geode query "MATCH (n:TestNode) DETACH DELETE n"

Transaction Rollback:

tests:
  - name: "isolated-test"
    setup:
      - begin
    args: ["query", "CREATE (n:TestNode {id: 1})"]
    expect:
      exit_code: 0
    teardown:
      - rollback

Troubleshooting

Common Test Issues

Test Server Won’t Start:

# Check if port is in use
lsof -i :3141

# Kill existing server
pkill -f "geode serve"

# Start server manually to debug
./zig-out/bin/geode serve --listen 127.0.0.1:3141 --log-level debug

Tests Timing Out:

# Increase timeout
export GEODE_TEST_TIMEOUT=60

# Run with longer timeout
python3 scripts/test/extended_test_runner.py --timeout 120

Inconsistent Test Results:

# Check for race conditions
# Run tests multiple times
for i in {1..10}; do
  python3 scripts/test/extended_test_runner.py --specs geodetestlab/specs/flaky_test.yml
done

# Use fixed seed for reproducibility
export GEODE_TEST_SEED=12345

Memory Leaks in Tests:

# Run with leak detection
valgrind --leak-check=full ./zig-out/bin/geode-test

# Or use Zig's testing allocator (automatically detects leaks)
zig test src/storage/heap.zig

Failed JSON Assertions:

# Enable verbose mode to see actual JSON
python3 scripts/test/extended_test_runner.py --verbose

# Manually inspect output
./zig-out/bin/geode query "..." --json | jq .

Debugging Strategies

Isolate Failing Test:

# Comment out other tests in spec file
tests:
  # - name: "test1" ...
  # - name: "test2" ...
  - name: "failing-test"
    args: ["query", "..."]

Add Debug Output:

tests:
  - name: "debug-test"
    args: ["query", "MATCH (n) RETURN n", "--verbose", "--log-level", "debug"]

Manual Reproduction:

# Start server
./zig-out/bin/geode serve --listen 127.0.0.1:3141 --log-level debug

# In another terminal, run exact command
./zig-out/bin/geode query "MATCH (n) RETURN n" --server 127.0.0.1:3141 --verbose

Use Interactive REPL:

# Start REPL for interactive testing
./zig-out/bin/geode shell

# Execute queries interactively
geode> MATCH (n:Person) RETURN n;
geode> .explain MATCH (n:Person) WHERE n.age > 30 RETURN n;

Test Infrastructure Issues

geodetestlab Import Errors:

# Install missing dependencies
pip3 install pyyaml

# Verify Python version
python3 --version  # Should be 3.9+

Test Runner Crashes:

# Enable debug mode
export GEODE_DEBUG=1
python3 scripts/test/extended_test_runner.py --verbose

# Check logs
tail -f /tmp/geode-test-runner.log

Server Not Responding:

# Check server status
ps aux | grep geode

# Check server logs
tail -f /tmp/geode-server.log

# Restart server
pkill -f "geode serve"
./zig-out/bin/geode serve --listen 127.0.0.1:3141 &

Best Practices

Test Writing Guidelines

Descriptive Names - Use clear, specific test names that describe what is being tested
Single Responsibility - Each test should verify one specific behavior
Independent Tests - Tests should not depend on execution order or shared state
Fast Tests - Keep unit tests fast (< 100ms), integration tests reasonable (< 5s)
Clear Assertions - Use specific assertions that clearly show what went wrong
Cleanup - Always clean up resources (use defer in Zig, teardown in geodetestlab)
Documentation - Add comments explaining complex test logic or non-obvious behavior

Test Organization

Group Related Tests - Organize tests by feature or component
Use Fixtures - Share test data through fixtures, not duplicated setup code
Parameterized Tests - Use data-driven testing for multiple similar scenarios
Hierarchical Specs - Structure geodetestlab specs by feature category
Naming Conventions - Follow consistent naming: component_feature_scenario

Continuous Improvement

Monitor Flaky Tests - Track and fix tests with inconsistent results
Review Coverage - Regularly review coverage reports and add tests for gaps
Update Tests - Keep tests current with code changes
Refactor Tests - Apply same code quality standards to test code
Document Known Issues - Maintain list of known test limitations in KNOWN_TEST_ISSUES.md

Next Steps

After mastering Geode testing:

Write New Tests - Add tests for new features or uncovered code paths
Optimize Performance - Use benchmark tests to validate optimizations
Contribute Tests - Submit test improvements via pull requests
Review Test Results - Monitor CI/CD test results and fix failures
Improve Coverage - Work toward higher test coverage in untested areas

Related Documentation:

Development Workflow - Contributing guidelines
CI/CD Pipeline - Continuous integration setup
Performance Testing - Query optimization testing
Deployment Testing - Production validation
Client Testing - Client library testing

Last Updated: February 2026 Test Coverage: 95.9% (3395/3540 extended suite tests passing) GQL Conformance Profile: ISO/IEC 39075:2024 compliance (see conformance profile) Status: Production Ready

Testing Guide Share link

Overview Share link

Test Coverage Summary Share link

Testing Frameworks Share link

Quick Start Share link

Running All Tests Share link

Running Specific Test Categories Share link

Test Environment Setup Share link

Zig Unit Tests Share link

Overview Share link

Writing Unit Tests Share link

Running Unit Tests Share link

Test Organization Share link

Common Patterns Share link

geodetestlab Framework Share link

Overview Share link

Test Specification Format Share link

Variable Substitution Share link

JSON Assertions Share link

Running geodetestlab Tests Share link

Test Organization Share link

Writing New Tests Share link

Debugging Failed Tests Share link

Integration with CI/CD Share link

Regression Tests Share link

Overview Share link

Available Regression Suites Share link

Writing Regression Tests Share link

Running Regression Tests Share link

Fuzz Testing Share link

Overview Share link

Fuzz Targets Share link

Running Fuzz Tests Share link

Fuzz Test Configuration Share link

Analyzing Fuzz Crashes Share link

Benchmark Tests Share link

Overview Share link

Benchmark Categories Share link

Running Benchmarks Share link

Benchmark Output Format Share link

Performance Regression Detection Share link

Test Coverage Analysis Share link

Measuring Coverage Share link

Coverage Reports Share link

Coverage Thresholds Share link

CI/CD Integration Share link

GitLab CI Pipeline Share link

GitHub Actions (Alternative) Share link

Test Artifacts Share link

Test Data Management Share link

Test Datasets Share link

Test Isolation Share link

Troubleshooting Share link

Common Test Issues Share link

Debugging Strategies Share link

Test Infrastructure Issues Share link

Best Practices Share link

Test Writing Guidelines Share link

Test Organization Share link

Continuous Improvement Share link

Next Steps Share link

Testing Guide

Overview

Test Coverage Summary

Testing Frameworks

Quick Start

Running All Tests

Running Specific Test Categories

Test Environment Setup

Zig Unit Tests

Overview

Writing Unit Tests

Running Unit Tests

Test Organization

Common Patterns

geodetestlab Framework

Overview

Test Specification Format

Variable Substitution

JSON Assertions

Running geodetestlab Tests

Test Organization

Writing New Tests

Debugging Failed Tests

Integration with CI/CD

Regression Tests

Overview

Available Regression Suites

Writing Regression Tests

Running Regression Tests

Fuzz Testing

Overview

Fuzz Targets

Running Fuzz Tests

Fuzz Test Configuration

Analyzing Fuzz Crashes

Benchmark Tests

Overview

Benchmark Categories

Running Benchmarks

Benchmark Output Format

Performance Regression Detection

Test Coverage Analysis

Measuring Coverage

Coverage Reports

Coverage Thresholds

CI/CD Integration

GitLab CI Pipeline

GitHub Actions (Alternative)

Test Artifacts

Test Data Management

Test Datasets

Test Isolation

Troubleshooting

Common Test Issues

Debugging Strategies

Test Infrastructure Issues

Best Practices

Test Writing Guidelines

Test Organization

Continuous Improvement

Next Steps