VIOVIO Survey BenchmarkGitHub Pages reproducible benchmark

Lesson 05 / Evaluate

Evaluate with EPA

Use EPICA / EPA as the active evaluation pipeline while preserving OpenVINS-style compatibility references.

Learning outcomes

  • Pair a `truths/` trajectory with an `algorithms/` trajectory
  • Know which EPA artifacts support a public row
  • Keep OpenVINS ov_eval as compatibility, not the primary publication path

Evaluation pair

EPA compares a truth trajectory against an estimated trajectory.

For the benchmark, those files are organized as `truths/<dataset>/<sequence>.txt` and `algorithms/<system>/<dataset>/<sequence>.txt`.

Public artifact

EPA outputs go under `epa_outputs/<system>/<dataset>/<sequence>/`.

A result row stays TBD until the EPA summary, command metadata, and run artifacts all point to the same case.

Checklist

  • Truth file exists
  • Estimated trajectory exists
  • EPA output folder exists
  • Alignment and timestamp association policy are recorded

Advanced notes

ATE, RPE, drift rate, and success rate depend on association and alignment policy. Publish that policy with the metric row.

OpenVINS-style commands are useful for compatibility checks, but EPA is the public result pipeline.

Practice task

Write the EPA input pair for one system, one dataset, and one sequence using `truths/` and `algorithms/` paths.