VIOVIO Survey BenchmarkGitHub Pages reproducible benchmark

Runtime and resources

Resource profiling is part of the benchmark

Accuracy rows are not enough for a reproducible VIO benchmark. Public results must also record timing, CPU, memory, platform metadata, and artifact status without inventing values.

Public artifacts

Profiling outputs

Resource outputs follow the same TBD-safe policy as accuracy metrics. Missing logs remain visible as status values, not hidden assumptions.

Timing CSV

results/time/<system>/<dataset>/<sequence>/run_time.csv

timestamp_s, frame_id, tracking_ms, estimation_ms, total_ms, status

Use TBD until a run-time log is generated from the recorded benchmark command.
CPU and memory monitor

results/resource/<system>/<dataset>/<sequence>/monitor_cpu_memory.csv

timestamp_s, cpu_percent, memory_mb, threads, platform_id, status

Use TBD until the process monitor and platform metadata are both present.
Platform metadata

results/resource/<system>/<dataset>/<sequence>/platform.json

cpu, memory, operating system, container tag, driver/runtime versions

Do not compare resource rows across different platforms without an explicit note.
EPA-linked summary

results/epa/<system>/<dataset>/<sequence>/metrics.json

accuracy metrics, association policy, alignment policy, output status

Resource rows are interpreted next to the EPA case status, not as standalone claims.

File schema

TBD-safe profiling templates

Templates define the column contract before numeric profiling results are released.

run_time.csv

timestamp_s,frame_id,tracking_ms,estimation_ms,total_ms,status
TBD,TBD,TBD,TBD,TBD,TBD

monitor_cpu_memory.csv

timestamp_s,cpu_percent,memory_mb,threads,platform_id,status
TBD,TBD,TBD,TBD,TBD,TBD