Primary trajectories are current-timestep odometry estimates.
Evaluation protocol
Rules before numbers
The benchmark is evaluated with EPICA / EPA and OpenVINS-style documentation discipline. The protocol defines valid trajectories, metric semantics, missing-data rules, and artifact requirements.
Protocol
Public evaluation rules
These rules are intentionally short and strict. They prevent the website from becoming a leaderboard with unclear inputs.
Systems with SLAM components must be configured for odometry-only reporting in this benchmark.
EPA is the primary evaluation pipeline for alignment, association, metric generation, plots, summaries, and paper tables.
OpenVINS ov_eval-style commands are compatibility references, not the primary publication path.
TBD, N/A, Partial, and Unresolved rows are visible but excluded from numeric averages unless a visible rule states otherwise.
Every public result row must carry system, dataset, sequence, mode, command source, artifact path, and missing-data status.
Metrics
Metric definitions and missing-data policy
The dashboard reports metric families only when their EPA or profiling artifacts are present.
Metric policy
| Metric | Name | Scope | Missing-data rule |
|---|---|---|---|
| ATE | Absolute Trajectory Error | Full trajectory after configured EPA alignment | TBD until metrics.json or summary.csv exists |
| RPE | Relative Pose Error | Local drift over configured distance, time, or frame pairs | TBD unless pair policy is recorded |
| Drift | Distance Drift Rate | Translation RPE normalized by path-length segment | TBD when segment policy is not available |
| Success | Drift-valid Success Rate | Reference-path percentage satisfying the configured criterion | TBD unless EPA drift-valid output is present |
| Runtime | Runtime | Tracking, estimation, and total per-frame timing | TBD until timing CSV is recorded |
| Resource | CPU and Memory | Process-level resource monitor logs on a labeled platform | TBD until monitor log is recorded |