Blog
Latest Posts
Section titled “Latest Posts”Agent Memory Benchmarks: Search, Context, Cache, and Snapshots
ZeptoDB Agent Memory benchmark results for filtered memory search, context assembly, exact cache lookup, semantic cache lookup, and sidecar snapshot persistence.
Read →Aggressive SIMD/JIT Optimization: From 2x to 4.2x Speedup
How ZeptoDB closed the gap with kdb+ using BitMask filtering, 4-way unrolled accumulators, and LLVM JIT O3 passes — with detailed benchmark results.
Read →ARM Graviton: 766/766 Tests Passing, 3x Faster GROUP BY
Full cross-platform verification of ZeptoDB on AWS Graviton (aarch64) — identical test results, Highway SIMD and LLVM JIT working natively, and surprising performance wins.
Read →Arrow Flight Server: High-Throughput Data Streaming
How ZeptoDB's Arrow Flight server enables remote Python, Polars, and DuckDB clients to stream query results over gRPC with near-zero-copy efficiency using the Arrow IPC protocol.
Read →Automatic Materialized View Query Rewrite
How ZeptoDB transparently redirects GROUP BY queries to pre-aggregated materialized views — turning O(n) partition scans into O(1) bucket lookups.
Read →Bandwidth Throttling and PTP Clock Sync for Distributed Clusters
How ZeptoDB prevents network saturation during rebalancing with a sliding-window rate limiter, and ensures ASOF JOIN correctness with PTP clock synchronization detection.
Read →Bare-Metal Performance Tuning for Microsecond Latency
A practical guide to CPU pinning, NUMA, huge pages, io_uring, and compiler optimizations — with real benchmark numbers from ZeptoDB on Intel Xeon.
Read →Building a Cost-Based Query Planner from Scratch
How ZeptoDB built a cost-based query planner in 7 phases — from HyperLogLog statistics to physical plan generation to EXPLAIN v2 with cost estimates.
Read →Building a SQL Parser, HTTP API, and JOIN Framework
How ZeptoDB implements a recursive descent SQL parser, ClickHouse-compatible HTTP API, and ASOF JOIN — the three pillars of a queryable time-series database.
Read →Building an Edition System: License Validation and Feature Gates
How ZeptoDB implements a two-tier edition system with JWT license tokens, offline RS256 validation, feature bitmask gating, HTTP 402 responses, and a trial key workflow.
Read →Cluster Integrity and Split-Brain Defense
How ZeptoDB prevents data corruption during network partitions using fencing tokens, epoch-based RPC validation, and Kubernetes lease integration.
Read →Cluster Stability Hardening: 15 Fixes for Production Readiness
A batch of critical fixes across ZeptoDB's distributed layer — thread pools, connection limits, 2PC snapshots, deadlock elimination, and state machine migrations — closing the gap between 'works in tests' and 'survives in production'.
Read →Coordinator HA and Automatic Failover Recovery
How ZeptoDB combines Kubernetes lease-based leader election with automatic re-replication to survive coordinator failures and data node losses without manual intervention.
Read →Data Durability: Intra-Day Snapshots and Crash Recovery
How ZeptoDB achieves sub-minute RPO for in-memory data with periodic snapshots, LZ4-compressed binary format, and single-threaded recovery before drain threads start.
Read →DDL and Schema Management in a Time-Series Database
Implementing CREATE TABLE, DROP TABLE, ALTER TABLE, and TTL-based retention in ZeptoDB — from schema registry to automatic partition eviction.
Read →Dictionary-Encoded String Columns for Time-Series
How ZeptoDB implements kdb+-style symbol interning with dictionary encoding — enabling WHERE symbol = 'AAPL' while maintaining integer-speed query performance.
Read →Distributed Cluster Architecture: Sharding, Routing, and Replication
How ZeptoDB scales horizontally with CRTP transport abstraction, consistent hashing, and UDP health monitoring — all with zero virtual call overhead on the hot path.
Read →Distributed Query Correctness: VWAP and ORDER BY Across Nodes
How ZeptoDB decomposes VWAP, HAVING, DISTINCT, and window functions for correct distributed execution using scatter-gather and partial aggregation.
Read →EKS Architecture Benchmark: amd64 vs arm64, 76/76 PASS
Full Kubernetes compatibility and HA/performance benchmarks on both amd64 and arm64 EKS — 76/76 tests passing, 7 bugs fixed, and arm64 confirmed production-ready with ~20% cost savings.
Read →Embedding DuckDB for Analytical Offload via Arrow
How ZeptoDB embeds DuckDB as an in-process analytical engine for Parquet queries — with Arrow columnar bridging and a duckdb() table function.
Read →Failover Data Recovery: Automatic Re-Replication
How ZeptoDB automatically re-replicates data when a node fails — PartitionMigrator integration, async recovery threads, and graceful fallback for unregistered nodes.
Read →Financial Functions for Time-Series: xbar, EMA, VWAP, and Window JOIN
Implementing kdb+-style financial analysis in SQL — xbar time bucketing, exponential moving averages, delta/ratio calculations, and Window JOIN for real-time analytics.
Read →FlatHashMap: Why We Replaced std::unordered_map for JOINs
How a custom open-addressing hash map with CRC32 hardware hashing eliminated pointer chasing and cache misses in ZeptoDB's JOIN operators.
Read →From Time-Series to Agent Memory: Real-Time Agents on ZeptoDB
ZeptoDB started as a microsecond time-series database. Agent Memory adds scoped recall, context assembly, prompt cache, and decision write-back so agents can reason over live data.
Read →Granular API Key Access Control
How ZeptoDB implements per-key symbol and table ACLs, tenant binding, key expiry, and in-place editing for fine-grained API key management.
Read →HDB Tiered Storage: From Memory to Parquet to S3
How ZeptoDB manages the hot/warm/cold data lifecycle with tiered storage — in-memory RDB for real-time queries, LZ4-compressed HDB on NVMe, and Parquet on S3 for long-term archival.
Read →Health Monitor: DEAD Recovery and UDP Fault Tolerance
How ZeptoDB's HealthMonitor handles DEAD node recovery via the REJOINING state, consecutive miss counting, TCP heartbeat fallback, and fatal bind failure detection.
Read →Helm Chart and Zero-Downtime Rolling Upgrades
How ZeptoDB's Helm chart enables zero-downtime rolling upgrades for an in-memory database — with PodDisruptionBudgets, config checksum annotations, canary deployments, and instant rollback.
Read →How ASOF JOIN Works in ZeptoDB
A deep dive into ZeptoDB's ASOF JOIN — from SQL parsing to SIMD-accelerated merge.
Read →HTTP Server Observability: Structured Logging and Request Tracing
How ZeptoDB's HTTP server provides production-grade observability — structured JSON access logs, slow query detection, X-Request-Id tracing, and Prometheus metrics.
Read →Internal RPC Security: Shared-Secret HMAC and mTLS
How ZeptoDB secures inter-node RPC communication with FNV-1a HMAC authentication, nonce-based replay protection, and a path toward mutual TLS.
Read →Introducing ZeptoDB
Why we built ZeptoDB: a source-available microsecond time-series database that now adds Agent Memory, prompt cache, and zero-copy Python.
Read →JIT SIMD Emit: Generating AVX2 Vector IR in LLVM
How ZeptoDB's JIT engine generates explicit <4 x i64> SIMD vector IR instead of relying on LLVM auto-vectorization — with mask extraction via cttz.
Read →Kubernetes Compatibility and HA Testing
How we built and executed a 38-test Kubernetes compatibility and high-availability test suite against a live EKS cluster — covering Helm validation, pod lifecycle, networking, rolling updates, and failure recovery.
Read →Kubernetes Operator with License-Aware Feature Gating
How ZeptoDB's bash-based K8s operator translates CRDs into Helm releases with license-aware multi-node gating and separation of concerns for JWT validation.
Read →Live Rebalancing: Zero-Downtime Partition Migration
How ZeptoDB moves partitions between nodes without data loss using dual-write ingestion, a state machine for migration lifecycle, and crash-recoverable checkpoints.
Read →Lock-Free Ingestion at 5.52M Events/sec
Inside ZeptoDB's MPMC ring buffer — design decisions, SIMD batch copy, and Highway vectorization.
Read →Parallel Query Engine: From Single-Core to Multi-Node
How ZeptoDB's QueryScheduler abstraction enables partition-level parallelism today and distributed execution tomorrow — without changing the executor.
Read →Parquet on S3: Historical Data Storage for Time-Series
How ZeptoDB writes Apache Parquet files and uploads them to S3 for long-term historical storage — with partition-based file layout, compression codec selection, and direct query integration with DuckDB and Polars.
Read →Python Ecosystem Integration: NumPy, Pandas, and Polars
How ZeptoDB integrates with the Python scientific stack — zero-copy Arrow paths, vectorized batch ingestion, and seamless DataFrame conversion for pandas, Polars, and DuckDB.
Read →Query Editor: Charts, Multi-Tab, Multi-Statement
Three enhancements to ZeptoDB's web query editor — interactive chart visualization with Recharts, multi-tab editing with localStorage persistence, and sequential multi-statement execution.
Read →RingConsensus: Lightweight Partition Ownership Protocol
Why ZeptoDB chose epoch broadcast over Raft for partition router synchronization, and how the pluggable consensus interface keeps the door open for stronger guarantees.
Read →SIMD-ifying the Window JOIN Aggregate Loop
How ZeptoDB applies a three-tier SIMD strategy to the Window JOIN aggregate loop — contiguous fast-path, gather+SIMD for large windows, and scalar fallback.
Read →SQL DML: INSERT, UPDATE, DELETE for Time-Series Data
Implementing mutation operations on columnar time-series data — INSERT with auto-timestamp, in-place UPDATE, and compaction-based DELETE.
Read →SQL Phase 2-3: Arithmetic, CASE WHEN, Date/Time, and Set Operations
Extending ZeptoDB's SQL with expression evaluation, CASE WHEN branching, temporal functions, LIKE pattern matching, and UNION/INTERSECT/EXCEPT set operations.
Read →SSO and JWT Authentication: From JWKS to Multi-IdP Support
How ZeptoDB evolved its authentication from static JWT keys to full SSO with JWKS auto-fetch, multi-IdP routing, group-to-role mapping, OIDC discovery, and token refresh.
Read →Storage Tiering and Materialized Views
Automatic hot/warm/cold/drop storage tiering and incremental materialized views for pre-computed aggregations in ZeptoDB.
Read →Vault-Backed API Key Store: Secrets Management for Databases
How ZeptoDB integrates HashiCorp Vault as a write-through backend for API keys, enabling multi-node key sharing and centralized secrets management.
Read →WAL Replication: Quorum Writes, Retry, and Backpressure
How ZeptoDB's WalReplicator evolved from best-effort async to production-grade replication with quorum acknowledgment, exponential retry, and configurable backpressure.
Read →Why Agent Memory Needs Time-Series Data
Agent memory is weaker when it is detached from the live event stream. ZeptoDB keeps memories, prompt cache, AgentOps telemetry, and time-series evidence on one replayable timeline.
Read →XBAR Sorted-Scan Optimization and EXPLAIN Statement
Exploiting timestamp monotonicity to eliminate hash lookups in GROUP BY XBAR, adding EXPLAIN for query plan inspection, and standardizing NULL handling across all executor paths.
Read →Zero-Copy Python: 522ns to NumPy
How ZeptoDB achieves sub-microsecond Python query results using memory-mapped column views.
Read →Release Notes
Section titled “Release Notes”| Version | Date | Highlights |
|---|---|---|
| — | — | Release notes will appear here as versions are published. |
Subscribe
Section titled “Subscribe”Stay updated on new posts and releases.
Newsletter signup coming soon.