VIOVIO Survey BenchmarkGitHub Pages reproducible benchmark

Lesson 07 / Evaluate

Datasets and ground truth

Use the SOP dataset order and truth-file structure before comparing systems.

Learning outcomes

  • Name the five SOP dataset targets
  • Know which datasets have first-stage GT handling
  • Understand why adapters are part of the benchmark

SOP dataset targets

The public benchmark targets EuRoC MAV, UZH-FPV, LaMAria, FLSea, and SubT-MRS / GrandTour.

EuRoC MAV and UZH-FPV are the first-stage datasets in the SOP because their GT paths are explicitly called out.

Adapters are not optional

Different datasets arrive as raw folders, ROS bags, or dataset-specific structures.

The adapter layer records the conversion into the benchmark input and truth layout.

Checklist

  • Dataset path is known
  • Truth path is known or marked Partial/TBD
  • Adapter path is known
  • Sequence list is not invented

Advanced notes

Dataset conversion can change timing, image order, frame convention, or calibration interpretation. Those choices must be captured before comparing rows.

Practice task

Open each dataset detail page and classify its GT status as TBD or Partial.