Frame records
A single observation carries pose, timing, units, provenance, ambiguity, diagnostics, and detections in the v1 schema.
V1 standalone audio-sensor package · Isaac Sim · Isaac Lab
Isaac Audio Sensors V1
A reusable microphone-array package for robotics simulation: stable AudioSensorFrame v1 records, deterministic geometry and TDOA backends, optional room-acoustics support, lazy Isaac Sim/Lab paths, Omniverse reference UX, and JSON/JSONL export.
- Package release
- 1.0.0
- Frame contract
- v1
- Stable backends
- L0/L1
- Exports
- JSON/JSONL
Final V1 package version promoted on 2026-05-24.
AudioSensorFrame schema version ias.audio_sensor_frame.v1.
geometry_only and tdoa_synthetic are stable V1 runtime levels.
Package-native frame and trace formats for V1 consumers.
capture.scope · four mic channels660 Hz event tone
frontrightrearleft
§ 01 — Overview
Overview
The package surface is defined first, then set beside adjacent simulator, Lab, acoustic, dataset, and synthetic-data tools from the technical report.
Deterministic core
The base backends produce repeatable bearings, sectors, delay, RMS, max-event behavior, and ambiguity metadata.
Runtime adapters
Isaac Sim, Isaac Lab, Omniverse, and trace export sit around the pure package instead of becoming hidden core dependencies.
§ 02 — Architecture
Architecture
The V1 architecture is deliberately conservative: the pure Python contract stays import-safe, while Isaac Sim, Isaac Lab, Omniverse, room-acoustics, Replicator, and downstream adapters remain optional surfaces.
Pure Python core
Scene snapshots, microphone arrays, time windows, detections, DOA estimates, schema export, and trace IO import without Isaac, Omniverse, ROS 2, CUDA, or room-acoustics dependencies.
Stable backends
geometry_only emits deterministic bearings and sector labels; tdoa_synthetic emits direct-path per-microphone delay and RMS diagnostics with explicit ambiguity metadata.
Isaac live paths
Lazy Isaac Sim and Isaac Lab helpers support stage discovery, live pose reads, SensorBase-compatible buffers, entity binding, and user-managed runtime validation.
Reference UX and export
The Omniverse extension is the reference authoring and inspection UX, while JSON/JSONL AudioSensorFrame output remains the stable package-native recording path.
Data flow
Scene state becomes an auditable AudioSensorFrame.
The technical report traces scene sources, array poses, time windows, backend diagnostics, and optional runtime provenance into the same V1 frame shape. JSON and JSONL remain the package-native export path for consumers and validation tools.
Scene snapshot
sources, array, poses, time window
Backend
geometry_only, tdoa_synthetic, optional room
Sensor frame
bearing, confidence, delay, RMS
Trace export
schema, JSON, JSONL, diagnostics
§ 03 — Evidence
Evidence
The evidence surface now favors V1-native artifacts: frame records, TDOA diagnostics, channel summaries, and the serialized frame contract.
artifact · audio_sensor_frames.jsonl
Four-channel trace summary
frames3 JSONL
arrayrig_front
baseline0.16 m
events3 scheduled
Pipeline · Capture to Bearing
V1 trace fields as an inspection flow
Capture
Capture: four mic buffers feed RMS + delay fields.
Correlate
Correlate: front->left TDOA peak at -0.330 ms.
Localize
Bearing: 44° from tdoa_synthetic, 0.948 conf.
artifact · trace JSON + frame JSONL
Bearing estimates against synthetic frame truth
bearing validation boardEstimate and synthetic truth stay nearly co-located
tdoa_synthetic01
front-rightSpeech · 0.00 s
0.009 deg44.1° estimate44.1° truth0.948 confidence
02
leftAlarm · 0.95 s
0.048 deg128.7° estimate128.7° truth0.943 confidence
03
rear-rightTool · 1.85 s
0.094 deg305.7° estimate305.8° truth0.937 confidence
Max absolute error0.094 deg
Mean confidence0.943
Backendtdoa_synthetic
| Event | Estimate | Truth | Error |
|---|---|---|---|
| speaker_front_right | 44.052 deg | 44.061 deg | 0.009 deg |
| speaker_left | 128.708 deg | 128.660 deg | 0.048 deg |
| speaker_rear_right | 305.744 deg | 305.838 deg | 0.094 deg |
contract · examples/traces/*.v1.json
Consumer-facing frame shape
// contract fields plus first public tdoa_synthetic detection
{
"schema_version": "ias.audio_sensor_frame.v1",
"backend_id": "tdoa_synthetic",
"array_id": "rig_front",
"sample_rate_hz": 48000,
"detections": [{
"source_id": "speaker_front_right",
"doa.estimated_bearing_deg": 44.05209380253932,
"doa.bearing_confidence": 0.9483431618821238,
"diagnostics.tdoa_matrix_s.front->left": -0.00032979783835574913
}]
}§ 04 — Reports
Reports
The site links the latest V1 technical report, the public trace assets, and the source repository that owns the package code and release scope.
PDF
V1 technical report
Paper-style report for isaac-audio-sensors 1.0.0 covering the frame contract, model, integrations, validation evidence, limitations, and future work.
Source
Public package repository
The package source, docs, examples, tests, and release notes live in the public GitHub repository.
§ 05 — Use cases
Use cases
These are package-supported workflows and natural downstream uses, not claims of real-world acoustic calibration or finished downstream validation.
V1 sensor traces
Use AudioSensorFrame records to inspect source pose, array pose, bearing, delay, RMS, ambiguity, provenance, and diagnostics.
ias.audio_sensor_frame.v1
Isaac Sim live sensors
Discover USD sound sources and arrays, read live transform stacks, update geometry_only or tdoa_synthetic frames, and write traces.
supported live path
Isaac Lab observations
Expose fixed-shape event_presence, bearing_deg, confidence, sector_onehot, per_mic_rms, and ambiguity_mask buffers for RL tasks.
SensorBase-compatible
Trace QA
Compare exported frames, schema files, event windows, DOA estimates, confidence values, and diagnostic namespaces before a release.
JSON + JSONL
Optional room acoustics
Run dependency-gated shoebox RIR, waveform, and GCC-PHAT diagnostics when the room extra is installed.
supported optional L2
Reference extension UX
Author arrays and sources, bind selected prims, select a backend, inspect overlays, and export package-native frames in Omniverse Kit.
extension-only Replicator option
§ 06 — Status
Status
The V1 release is a bounded sensor-frame and integration package. It does not claim complete acoustic realism, real hardware validation, sim-real transfer, or official NVIDIA product status.
Core package validation
The V1 release validates frame schema parity, JSON/JSONL trace round trips, stable L0/L1 behavior, optional L2 behavior, import smoke, lint, build, and distribution audit.
Isaac Sim live path
The final local 1.0.0 run emitted valid AudioSensorFrame v1 JSONL records for geometry_only and tdoa_synthetic in Isaac Sim 5.1.0 / Kit 107.3.3.
Isaac Lab live path
The final local GPU run recorded SensorBaseCfg/SensorBase subclass checks, fixed-shape RL buffers, stable observation keys, CUDA tensors, and stage/entity binding evidence.
Room acoustics backend
The room extra gates the room_acoustics backend behind pyroomacoustics and records L2 RIR, waveform, GCC-PHAT, TDOA, and per-mic RMS diagnostics when available.
Bounded claims
V1 is intentionally useful without overstating what the evidence proves.
Deterministic, not physical
geometry_only and tdoa_synthetic prove repeatable frame behavior; they do not model propagation, reverberation, or occlusion.
Acoustics stay dependency-gated
room_acoustics adds shoebox RIR, waveform, GCC-PHAT, and RMS diagnostics when installed, not calibrated material or microphone realism.
Higher fidelity remains future work
L3 advanced realism and L4 sim-real calibration are documented directions, not complete V1 runtime systems.
No downstream proof required
V1 does not claim hardware benchmarks, official NVIDIA product status, or mandatory Alex, SquadBot, and ROS 2 adapter validation.