Seonglae ChoFound that there is no statistically significant correlation between visual realism and physical understanding
While existing benchmarks relied on synthetic data and faced real-vs-synthetic distribution shift issues, Physics-IQ uses real, high-quality filmed videos. The Physics-IQ dataset consists of 66 scenarios captured from 3 camera viewpoints, filmed twice, totaling 396 videos (3840x2160, 30FPS)
Spatiotemporal IoU considers the time axis and compares motion masks on a frame-by-frame basis
Weighted Spatial IoU measures how much motion occurred at specific locations using a weighted average
The Physics-IQ score combines these 4 metrics and is normalized to 100% based on physical variance (the difference between two recordings of the same scenario).
