Physical AI & Action-Outcome Memory

Physical AI agents need a memory of the physical world, not only a memory of text. A robot decision depends on what the robot observed, which action was taken, whether a human intervened, which policy context applied, and what happened afterward.

ZeptoDB keeps high-frequency robot telemetry and agent memory on the same timeline. The product surface available today covers ROS 2 ingest, typed profile tables, rosbag2 import/replay, time-series SQL, ASOF JOIN, Window JOIN, zero-copy Python, and Agent Memory. The more specific context-gated Physical AI Action-Outcome Memory experiments are research-only; they are useful evidence for the product direction, not a claim that ZeptoDB is a production robot controller or certified safety system.

What ZeptoDB does for Physical AI

Record the situation

Capture robot state, sensor summaries, site, zone, payload, human proximity, operator interventions, and incident markers as ordered time-series evidence.

Remember the action

Store the retrieved context, policy note, proposed action, executed action, rollback path, outcome, and reflection as agent memory.

Gate the reuse

Let applications check whether a prior successful action still fits the current temporal, spatial, payload, and policy context.

Replay the decision

Reconstruct the chain from sensor window to retrieved memory, recommendation, suppression, intervention, outcome, and follow-up note.

The Action-Outcome loop

robot state + sensor window
  -> retrieved prior episodes
  -> policy and context gate
  -> recommended or suppressed action
  -> observation window
  -> outcome and reflection

For robots, a similar memory is not enough. The old outcome has to be checked against the current physical situation: zone, topology, payload, human proximity, environment, sensor motif, current policy, and whether a human approval path is required.

ZeptoDB provides the data substrate for that loop. It does not decide the action by itself. Your application, model, controller, or operator workflow remains responsible for policy and execution.

Safety-adjacent workflows

Use these as cautious product claims. ZeptoDB helps store and replay the evidence that robot applications need for guardrails; it is not a certified safety layer.

Risky-repeat suppression

Store why a prior action was suppressed when the current payload, zone, sensor motif, or policy context no longer matched the old successful episode.

Operator intervention memory

Keep human override, manual recovery, rollback, and inspection notes beside the telemetry that triggered the intervention.

Policy-aware recall

Attach allowed-action metadata, human-approval requirements, risk tier, geofence, and site policy context to each action-outcome episode.

Incident and near-miss replay

Ask what the robot saw, which memory was retrieved, whether an old action was reused or blocked, and what changed afterward.

Available today vs research-only

Status	Physical AI scope
Available today	ROS 2 scalar/profile ingest, typed profile tables, rosbag2 import/replay, time-series SQL, ASOF JOIN, Window JOIN, Agent Memory, exact/semantic cache, replay telemetry, zero-copy Python
Research-only	Context-gated Physical AI Action-Outcome Memory experiments, synthetic robot/fleet fixtures, SQL replay harnesses, edge/fleet replay shape
Not claimed	Fully autonomous robot control, certified robot safety, runtime edge-to-fleet replication, or a production control plane that executes robot actions

Read the grounded overview: Physical AI Memory and Action-Outcome Memory.

Why Physical AI teams choose ZeptoDB

522ns to tensor

Zero-copy path from column store to NumPy and PyTorch. Online inference and offline evaluation can share the same SQL and the same data.

Nanosecond ASOF JOIN

Time-align heterogeneous streams such as camera frames, IMU, tactile, proprioception, odometry, and LaserScan telemetry.

5.52M events/sec

Ingest high-frequency robot and edge telemetry with a lock-free MPMC ring buffer and zero allocation on the hot path.

Episodes on S3

Replay Parquet episodes for offline evaluation, imitation-learning datasets, regression tests, and incident review.

Action memory

Store retrieved context, operator interventions, policy notes, tool calls, action outcomes, and reflections beside the raw sensor stream.

ROS 2 bridge

Subscribe to scalar topics, standard ROS 2 profiles, or typed profile tables; import and replay the same profiles from rosbag2.

Where it fits in the stack

Action-Outcome Memory for robot decisions

Store the action the agent proposed, the action that was allowed or suppressed, and the outcome that followed
Compare similar robot incidents against current topology, temporal motif, and change context before repeating a historical action
Keep edge-local suppression decisions separate from fleet-global audit and policy learning
Replay why a proposal was allowed, suppressed, or escalated to manual review

Sensor fusion for perception

Direct ROS 2 topic ingest for scalar std_msgs, IMU, JointState, Odometry, TF, and LaserScan telemetry
ASOF JOIN across LiDAR point clouds, camera frames, IMU, and depth streams
Window JOIN with ±N ms tolerance for clock drift across heterogeneous buses
Nanosecond-precision timestamps end-to-end — parser to column store

Action-outcome memory for robot behavior

Store memory records for failed grasps, successful recoveries, operator corrections, risky-repeat suppressions, and safety interventions
Retrieve similar episodes before an agent recommends a plan
Compare old outcomes against current payload, site, zone, sensor pattern, and policy context
Replay the chain from sensor data to retrieved context to action outcome

Fleet telemetry and diagnostics

Ingest telemetry from thousands of robots or drones simultaneously
Detect anomalies with window functions for motor current, joint torque, battery, temperature, and localization drift
Keep historical forensics in Parquet HDB when something goes wrong in the field

Sim-to-real evaluation

Treat simulation rollouts and real-robot episodes as compatible schemas
Compare policy performance across batches with standard SQL aggregates
Version datasets with immutable Parquet snapshots

Action-Outcome Memory →

Current Action-Outcome evidence

ZeptoDB’s Physical AI research track now has concrete action/outcome evidence behind it:

15/15 hazardous proposals suppressed

Shadow supervisor tests blocked every hazardous baseline proposal and routed those decisions to manual review.

5/5 safe proposals allowed

Context-gated recovery proposals were allowed across the five robot incident families in the fixture.

52/52 edge/fleet ACK convergence

Bounded edge-to-fleet replay handled duplicate, late, outage, and restart cases while converging fleet audit rows.

Controlled pilot boundary

The SQL/HTTP adapter is admin-gated and pilot-scoped, with explicit non-goals and rollback requirements.

Research evidence →

ROS 2 ingest, typed profiles, and replay

ZeptoDB includes an optional ROS 2 connector for Physical AI pipelines. Build with -DZEPTO_USE_ROS2=ON and provide rclcpp, std_msgs, standard ROS 2 message packages, and optionally rosbag2_cpp / rosbag2_storage.

Live subscriptions currently support scalar std_msgs/msg/{Float64,Float32,Int64,Int32,UInt64,UInt32} topics using a single data field.
Ros2IngestMode::StandardProfile supports IMU, JointState, Odometry, TFMessage, and LaserScan profiles by flattening configured numeric fields into scalar rows.
Ros2IngestMode::TypedProfile writes schema-aware wide rows for the same standard profiles, including robot/session/topic/frame metadata and profile-specific numeric columns.
Typed profile rows participate in table-scoped cluster routing through typed-row RPC forwarding when the partition owner is remote.
rosbag2 import and replay use the same configured subscriptions as topic allowlists, preserving bag send timestamps as source time.
The bridge is read-only: it subscribes to ROS 2 data and imports bags without publishing back into the ROS graph.

Start with the ROS 2 Setup and Smoke Test, the ROS 2 Edge Deployment Guide, and the C++ ROS 2 connector reference.

Action-Outcome Memory research

The Action-Outcome Memory research line asks whether a robot should reuse a prior action only when the current temporal, spatial, payload, and policy context is compatible with the old outcome.

Current public evidence:

Experiment 013 compared similar robot incident retrieval, runbook/action priors, reflection-only memory, and context-gated Physical AI Action-Outcome Memory on a synthetic fixture. The context-gated variant reached 1.00 recovery Top-1 hit rate, 1.00 risky-repeat avoidance, and 0.00 hazardous top-action rate on that fixture.
Experiment 014 replayed the same Physical AI memory shape through live ZeptoDB SQL tables, including action/outcome JOINs, robot and sensor ASOF JOINs, ROW_NUMBER/LAG windows, suppression audit JOINs, and a spatial ST_Within check.
Experiment 015 modeled an edge-local and fleet-global split with two live ZeptoDB endpoints. This was a research harness, not a shipped runtime replication feature.

These are research fixture results, not production safety guarantees.

Example: sensor fusion pipeline

-- Align LiDAR scans with the most recent IMU sample and camera frame.
SELECT
    l.ts,
    l.point_cloud,
    i.accel_x, i.accel_y, i.accel_z,
    i.gyro_x,  i.gyro_y,  i.gyro_z,
    c.frame_id
FROM lidar_scans l
ASOF JOIN imu_readings  i ON l.robot_id = i.robot_id AND l.ts >= i.ts
ASOF JOIN camera_frames c ON l.robot_id = c.robot_id AND l.ts >= c.ts
WHERE l.ts > now() - interval '10 seconds'

import zeptodb
import torch

conn = zeptodb.connect("localhost:8123")

# Query -> NumPy -> Tensor in microseconds, no serialization.
result = conn.query(
    "SELECT accel_x, accel_y, accel_z FROM imu_readings WHERE ts > now()-1s"
)
tensor = torch.from_numpy(result.to_numpy())  # zero-copy view

Integration points

System	Integration
ROS 2 scalar topics	Available — live `std_msgs` subscriptions, table-aware ingest, rosbag2 import/replay
ROS 2 standard profiles	Available — IMU, JointState, Odometry, TFMessage, LaserScan scalar mapping
ROS 2 typed profiles	Available — schema-aware wide tables with typed-row cluster forwarding
MQTT	Ingestion connector for lightweight robot telemetry
OPC-UA	Industrial connector for co-bot and fixed-robot cells
PyTorch / JAX	Zero-copy via NumPy bridge
Arrow Flight	Batch streaming for distributed training
Parquet on S3	Immutable episode storage for dataset versioning

Start from GitHub

Build from the public repo, or open an issue with your pipeline shape: GitHub → · Issues →